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FOREWORD 


The  Infantry  Forces  Research  Unit  of  the  U.S.  Army  Research  Institute  for  the  Behavioral 
and  Social  Sciences  conducts  research  under  a  Science  and  Technology  Objective,  Training 
Objective  Force  Small  Unit  Leaders  and  Teams.  The  aim  of  the  research  is  to  design  efficient, 
effective  cognitive  skills  training  to  better  facilitate  rapid  and  accurate  decision  making.  In 
essence,  this  research  leverages  advances  in  situational  awareness  training  and  measurement, 
tailorable  training,  computer  gaming,  and  intelligent  tutoring.  As  part  of  this  objective,  our 
behavioral  scientists  work  closely  with  other  agencies  and  organizations  to  identify  new  training 
methods  and  technologies  being  developed  and  used  in  the  private  sector.  We  examine  these 
methods  and  techniques  for  their  usefulness  in  training  leader  cognitive  skills  and  in  developing 
and  validating  methods  and  measures  necessary  to  assess  leader  decision  making  and 
information  utilization. 

Recently,  we  were  asked  to  participate  in  the  design  and  execution  of  field  research  to 
evaluate  the  training  effectiveness  of  a  personal  computer-based  video  game  named  Full 
Spectrum  Command  (FSC).  The  game  was  developed  for  the  Army  by  the  Institute  for  Creative 
Technologies  at  the  University  of  Southern  California,  with  resources  and  expertise  provided  by 
both  technology-driven  talent  from  the  entertainment  software  industry  and  military  personnel 
from  the  U.S.  Army  Infantry  School.  The  objectives  of  the  game  were  to  let  prospective 
company  commanders  in  the  Infantry  Captains  Career  Course  conduct  mission  analysis  and 
plannings  experience  the  decision-making  requirements  that  occur  during  mission  execution,  and 
enhance  their  ability  to  adapt  to  emerging  conditions  on  a  simulated  battlefield. 

Shortcomings  in  experimental  procedures  confounded  the  measurement  of  adaptive 
decision,  making  it  difficult  to  gauge  behavioral  differences  between  officers  who  played  FSC 
and  those  who  did  not.  However,  other  results  suggest  that  playing  the  game  provided  tactical 
experiences  with  potential  training  value.  Officers  who  played  FSC  identified  specific  strengths 
in  the  game  and  suggested  changes  that  should  be  made  in  future  versions.  Measures  used  to 
quantify  the  prior  military  experience  and  decision-making  style  of  officers  were  shown  to  be 
related  to  their  opinions  about  the  training  value  of  FSC,  as  well  as  to  measures  of  adaptive 
decision-making  behaviors. 

Results  of  this  research  were  briefed  to  the  Director  of  the  Combined  Arms  &  Tactics 
Directorate,  the  Chief  of  the  Tactics  Department,  and  other  key  individuals  at  the  U.S.  Army 
Infantry  School,  as  well  as  to  individuals  associated  with  the  Center  for  Creative  Technologies. 
They  also  were  presented  at  meetings  of  the  behavioral  science  community  and  at  a  major 
training  games  symposium.  We  believe  our  results  will  influence  the  development  of  future 
training  methods  and  tools  for  assessing  decision-making  behaviors. 


FRANKLIN  L.  MOSES 


Acting  Technical  Director 
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TRAINING  EFFECTIVENESS  EVALUATION  OF  THE  FULL 
SPECTRUM  COMMAND  GAME 

EXECUTIVE  SUMMARY _ _ 


Research  Requirement: 

The  U.S.  Army’s  transformation  strategy  includes  new  force  capabilities  that  will  assure 
dominance  across  the  foil  spectrum  of  military  operations.  These  capabilities  will  also  require 
that  combat  leaders  make  rapid  adaptive  decisions  in  response  to  changing  information  about 
battlefield  conditions.  This  requirement  has  increased  demands  for  new  training  methods  to 
develop  the  cognitive  skills  required  by  future  leaders.  Recent  popular  news  accounts  have 
begun  to  emphasize  the  potential  value  of  video  games  for  training  cognitive  skills  and  decision¬ 
making  behaviors.  This  report  documents  the  approach  and  results  of  research  to  evaliiate  the 
training  effectiveness  of  one  such  video  game  named  Full  Spectrum  Command  (FSC). 

Procedure: 

Fifty-four  officers  enrolled  in  the  Infantry  Captains  Career  Course  at  Fort  Benning, 
Georgia,  served  as  participants  in  this  research.  One-half  were  assigned  to  a  group  that  played 
FSC  in  addition  to  participating  in  the  regularly  scheduled  course  work  for  commanding  a  light 
Infantry  company  during  urban  offensive  operations.  The  others  received  only  the  course  work. 
Pre-FSC  measures  were  obtained  of  the  officers’  experience  in  military  service,  general 
cognitive  ability,  and  decision-making  style.  The  conduct  of  the  class  and  the  behavior  of 
officers  in  the  FSC  group  were  recorded  during  their  train-up  and  during  their  subsequent 
playing  of  FSC.  A  questionnaire  administered  to  officers  in  the  FSC  group  documented  their 
sense  of  personal  involvement  in  the  simulated  mission,  perceptions  of  FSC  training  value,  and 
opinions  of  the  strengths  and  weaknesses  of  the  game.  Following  the  period  for  playing  FSC, 
officers  in  both  groups  were  re-administered  the  decision-making  style  inventory  and  assessed 
individually  for  their  tactical  decision-making  behaviors  as  the  commander  of  a  light  Infantry 
company  in  a  tactical  exercise  using  the  Janus  simulation.  A  Janus  Scenario  Assessment 
Checklist  was  used  to  estimate  each  officer’s  ability  to  adapt  to  changing  and  uncertain 
battlefield  conditions  during  the  Janus  exercise. 

Findings: 

Several  shortcomings  in  experimental  procedures  used  for  playing  FSC  confounded 
comparisons  of  adaptive  decision-making  behaviors  between  officers  who  played  FSC  and  tiiose 
who  did  not.  However,  other  results  suggest  that  playing  FSC  provided  tactical  experiences  that 
had  potential  training  value.  Officers  who  played  FSC  identified  specific  strengths  in  the  game 
as  well  as  changes  that  should  be  made  in  foture  versions.  Measures  used  to  quantify  the  prior 
military  experience  of  officers  were  related  to  their  perceptions  of  personal  involvement  in  and 
training  value  of  FSC,  their  adaptive  decision-making  behaviors  in  Janus,  and  their  decision¬ 
making  style. 
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Utilization  of  Findings: 


The  results  of  this  research  will  influence  the  future  development  of  training  games  and 
tools  for  assessing  decision-making  behaviors.  Findings  were  discussed  with  key  individuals 
from  the  training  community  at  Fort  Benning,  as  well  as  the  commercial  training 
technologies  industry.  Descriptions  of  the  research  have  also  been  presented  at  meetings  of  the 
behavioral  science  commimity  and  at  a  symposium  devoted  to  the  training  value  of  games. 
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INTRODUCTION 


Background 

The  U.S.  Amy’s  transformation  strategy  includes  new  force  capabilities  that  will  assure 
dominance  across  the  foil  spectrum  of  military  operations.  However,  experience  has  shown  that 
the  best  capabilities  cannot  support  command  and  control  of  the  force  without  trained  personnel. 
With  infomation  available  to  all  levels  of  the  command  and  increasing  dispersion  on  the 
battlefield,  junior  leaders  may  find  themselves  operating  almost  autonomously  (Department  of 
the  Amy,  DA;  2003).  In  the  absence  of  detailed  orders  to  cover  possible  battlefield 
contingencies,  these  leaders  need  to  be  trained  to  accept  risk,  seize  the  initiative  to  counter 
imforeseen  enemy  actions,  and  modify  plans  quickly  when  conditions  change.  Required  training 
must  increase  the  adaptability  or  flexibility  of  the  leader’s  decision-making  perfomance. 

Typically,  Amy  training  utilizes  the  crawl-walk-run  approach  (DA,  2002).  This  method 
begins  at  the  basic  level  in  which  training  events  are  relatively  simple  and  resource  requirements 
are  minimal.  However,  training  at  this  level  can  sometimes  lack  realism.  As  the  difficulty  and 
complexity  of  training  events  increase,  there  is  a  requirement  for  more  resources,  but  the  level  of 
realism  can  increase  to  simulate  conditions  expected  in  a  combat  environment.  Those 
responsible  for  training  small  unit  leaders  rely  upon  a  mix  of  techniques  to  accomplish  their 
training  objectives.  Interest  in  using  relatively  low-cost  simulators  to  emulate  realistic  combat 
environments  has  increased  along  with  the  sophistication  and  power  of  personal  computers.  This 
report  addresses  one  effort  to  use  a  personal  computer-based  combat  simulation  to  help  train 
future  company  commanders  in  the  Army. 

The  effort  was  part  of  a  developmental  tasking  by  the  U.S.  Army  Program  Executive  Office 
-  Simulations,  Training  &  Instrumentation  (PEO-STI^  to  the  Institute  for  Creative 
Technologies  (ICT)  at  the  University  of  Southern  California.  This  tasking  focused  on  creating 
computer  “games”  to  be  used  as  learning  tools  for  the  cognitive  development  of  small  unit 
leaders.  To  accomplish  this  task,  the  ICT  went  to  the  entertainment  software  industry  for 
resources  and  expertise.  One  of  the  resulting  products  was  a  personal  computer-based  game 
dubbed  Full  Spectrum  Command  (FSC).  This  game  was  designed  for  use  in  the  Infantry 
Captains  Career  Course  (ICCC)  at  Fort  Benning,  Georgia.  It  let  students  in  the  course, 
prospective  company  conunanders,  conduct  mission  analysis  and  planning,  and  it  then  enabled 
them  to  experience  the  decision-making  requirements  that  occm  during  mission  execution.  A 
specific  additional  purpose  of  the  simulation  was  to  sharpen  the  students’  ability  to  adapt  to 
emerging  threats  and  changing  conditions  on  the  simulated  battlefield.  The  Infantry  Forces 
Research  Unit  of  the  U.S.  Army  Research  Institute  (ARI)  at  Fort  Benning,  Georgia,  was  asked 
by  PEO-STRI  to  conduct  an  empirical  evaluation  of  FSC.  The  U.S.  Army  Research 
Development  and  Engineering  Command  —  Simulation  Technology  Center  (RDECOM-STC) 
was  responsible  for  monitoring  this  effort.  This  research  report  documents  the  approach  taken 
and  the  results  of  this  evaluation. 


*  For  ease  of  scientific  discourse,  the  officers  enrolled  in  the  ICCC  who  participated  in  this  evaluation  research  are 
called  alternatively  students,  participants,  and  players.  Because  all  the  officers  were  males,  they  are  referred  to  by 
using  only  masuline  pronouns. 
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The  FSC  Game 


FSC  is  a  computer-based  simulation  of  an  urban  environment  in  which  a  U.S.  Army 
Captain  commands  a  light  Infantry  company  offensive  operation.^  The  simulated  terrain  in  FSC 
represents  a  1 -kilometer  square  area  modeled  roughly  after  the  McKenna  Military  Operations  in 
Urban  Terrain  (MOUT)  Site  at  Fort  Benning,  Georgia.  Blue  force  structures  portrayed  in  FSC 
simulate  those  available  to  a  typical  light  Infantry  company  commander.  The  enemy  force 
represented  in  the  game  is  the  kind  of  asymmetric  force  that  might  be  encountered  in  typical 
urban  fighting.  It  consists  of  loosely  organized  small  groups  of  men  who  possess  AK47A  rifles, 
rocket  propelled  grenades,  and  sniper  rifles. 

Early  in  the  development  of  FSC,  subject  matter  experts  were  drawn  fi-om  the  pool  of 
small  group  instructors  for  the  ICCC.  These  subject  matter  experts  provided  input  into  the 
development  of  the  artificial  intelligence  software  that  determined  doctrinally  correct  actions  by 
blue  force  and  enemy  force  elements  in  the  simulated  battlefield  environment  In  addition,  since 
they  recognized  the  potential  value  of  a  company  command  simulation,  the  subject  matter 
experts  ensured  that  FSC  was  developed  and  structured  to  meet  training  needs  of  the  ICCC. 
However,  constraints  during  the  development  of  FSC  did  not  allow  the  implementation  of  some 
battlefield  assets  such  as  combined  arms,  close  air  support,  engineers,  and  mechanized  forces.  In 
spite  of  these  limitations,  the  game  provided  small  group  instructors  at  the  ICCC  with  another 
tool  to  train  company  commander  skills. 

FSC  presents  a  battalion  operations  order  to  the  student  who  is  plajdng  the  game.  A 
series  of  computer  monitor  screens  facilitate  the  player’s  analysis  of  the  assigned  mission,  his 
organization  of  available  assets  across  maneuver  units,  and  his  creation  of  a  mission  execution 
matrix  that  synchronizes  the  performance  of  required  tasks  by  the  maneuver  units  over 
successive  phases  of  the  mission.  Once  these  required  planning  steps  have  been  completed,  the 
player  can  initiate  action  and  begin  the  experience  of  observing  and  responding  to  emerging 
conditions  portrayed  in  the  simulated  battlefield.  The  capability  of  FSC  to  permit  the  player  to 
respond  to  emerging  battlefield  conditions  is  a  unique  feature  of  FSC.  The  player  can  issue 
fiagmentary  orders  to  change  his  original  plan  if  it  becomes  irrelevant  due  to  emerging  events. 
Consequently,  FSC  permits  the  player  to  practice  the  behaviors  necessary  to  acquire  and 
maintain  adaptive  decision-making  skills  in  a  realistic  and  dynamic  tactical  environment.  FSC 
tracks  and  records  data  automatically  that  describe  what  happens  during  the  execution  of  a 
mission.  The  accessibility  of  those  data  and  the  capability  of  FSC  to  replay  specified  portions  of 
the  mission  provide  opportunities  for  a  player  and  an  instructor  to  examine  the  mission  in  detail. 
This  performance  feedback  facilitates  the  proper  development  of  the  player’s  adaptive  command 
and  control  skills. 


Instructional  Characteristics  of  the  ICCC 

The  ICCC  program  of  instruction  is  quite  full,  leaving  little  room  for  any  deviation  fi-om  the 
prescribed  schedule  of  instructional  events.  The  typical  class  is  about  18  weeks  in  duration.  The 
first  ten  weeks  are  devoted  to  instruction  and  practical  exercises  designed  to  develop  the 
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A  User  Manual  for  FSC  provides  a  detailed  description  of  the  game.  An  electronic  copy  of  the  User  Manual,  in 
Adobe  PDF  format,  is  available  from  the  authors. 
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cognitive  skills  needed  to  command  and  control  a  company-size  tactical  unit.  The  goal  of  this 
section  of  the  class  is  for  the  student,  acting  as  a  company  commander,  to  issue  an  operations 
order  for  an  Infantry  company.  Subsequent  sections  of  the  class  are  devoted  to  instruction  and 
practical  exercises  designed  to  develop  the  student’s  ability  to  serve  as  a  company-grade 
battalion  or  brigade  staff  officer  (six  weeks),  to  plan  and  prepare  training  as  a  company 
commander  or  as  an  assistant  staff  officer  (one  week),  and  to  participate  in  focused  experiential 
exercises  to  expand  their  knowledge  about  specific  type  of  orgamzation  in  which  they  will  be 
commanding  a  company  (one  week). 

Beginning  around  the  fifth  week  in  the  class,  the  students  begin  a  six  to  seven  ^y  lesson 
devoted  to  commanding  a  light  Infantry  company  in  urban  offensive  operations.  This  particular 
lesson  stresses  urban  terrain  analysis,  the  understanding  of  weapons  effects,  integration  of  non- 
organic  forces,  enemy  defensive  tactics  in  urban  areas,  and  specific  planning  considerations  for 
urban  operations.  The  lesson  is  designed  to  reinforce  the  mission  analysis,  course  of  action 
development,  and  course  of  action  analysis  processes  of  the  troop  leading  procedures.  Its 
terminal  learning  objective  is  for  the  student  to  demonstrate  an  improvement  of  his  visualization 
of  the  tactical  problem  and  an  ability  to  describe  his  plan  for  executing  and  synchronizing  a  light 
company  course  of  action  in  urban  operations.  The  standard  for  demonstrating  these  skills  is  the 
student’s  performance  while  he  briefs  his  mission  analysis  and  course  of  action.  An  after  action 
review  of  the  briefing  consists  of  interactive  discussions  among  the  students  and  die  small  group 
instructor. 

The  ICCC  has  not  been  able  to  provide  realistic  experiences  for  the  adaptive  decision¬ 
making  necessary  to  respond  to  changes  that  occur  in  the  threat  or  other  battlefield  conditions. 
Likewise,  students  have  not  had  repeated  opportunities  to  practice  and  actually  experience  the 
consequences  of  executing  their  operational  plans  or  changing  their  plans  in  response  to 
emerging  conditions  on  the  battlefield.  FSC  was  designed  and  developed,  in  part,  to  fill  these 
gaps  in  student  training  and  experience. 

Evaluation  Concents  and  Objectives 

We  recognized  that  opportunities  for  students  to  play  FSC  would  occur  in  the  context  Of  an 
ongoing  program  of  instruction.  This  was  to  be  an  instance  of  field  research  and  not  controlled 
experimental  research.  During  initial  meetings  between  the  authors  and  the  leadership  of  the 
ICCC,  the  need  was  stressed  for  the  maximum  possible  control  over  procedures  for  using  FSC  in 
the  course  and  for  the  use  of  multiple  methods  to  assess  the  potential  impact  of  FSC  on  the 
command  abilities  of  the  students.  As  preparation  for  this  evaluation  research  began,  it  was 
agreed  by  all  interested  parties  that  the  research  objectives  would  include  the  following  sets  of 
analyses: 

(1)  The  principal  measure  of  the  training  effectiveness  of  FSC  on  the  acquisition  of 
adaptive  decision-making  skills  would  be  derived  fi-om  measured  comparisons  between  students 
assigned  to  a  group  that  played  FSC  and  those  assigned  to  a  control  group  that  did  not  play  FSC. 

(2)  A  second  basis  for  evaluating  the  training  impact  of  FSC  on  adaptive  decision  making 
would  be  observations  and  ratings  of  students  who  played  FSC  by  ICCC  small  group  instructors. 
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The  students’  responses  to  a  questionnaire  designed  to  record  their  perceptions  and  opinions  of 
FSC  would  the  aid  evaluation. 

(3)  Other  data  pooled  over  students  in  both  the  FSC  and  control  groups,  while  unrelated  to 
the  impact  of  FSC  per  se,  would  be  analyzed  for  their  potential  value  to  those  responsible  for 
training  at  the  ICCC  and  to  other  Army  leaders. 


METHOD 

Participants 

Students  enrolled  in  two  different  seminars  for  each  of  two  successive  classes  of  the  ICCC 
served  as  participants  in  this  evaluation  research.  In  each  class,  students  in  one  seminar  were 
assi^ed  to  participate  in  the  FSC  group  and  those  in  the  other  seminar  were  assigned  to 
participate  in  the  control  group.  There  were  14  students  in  each  seminar  during  the  first  class 
and  13  in  each  seminar  during  the  second  class,  27  participants  for  each  group.  Other  than 
having  different  small  group  instructors  directing  the  activities  of  their  seminars,  the  students 
fi-om  the  four  different  seminars  were  presented  the  same  program  of  instruction  with  one 
exception.  Students  in  the  FSC  groups  were  given  opportunities  to  play  FSC  over  a  period  of 
about  seven  class  days,  while  those  in  the  control  group  had  no  instruction  or  hands-on 
experience  with  FSC. 


Measurement  Instruments 
Biographical  Information  Questionnaire 

A  <^py  of  the  Biographical  Information  Questionnaire  is  presented  on  pages  A-2  and  A-3 
of  Appendix  A.  This  questionnaire  was  designed  by  the  authors  to  permit  each  student  to 
describe  the  types  and  breadth  of  experiences  he  had  that  might  be  relevant  to  this  evaluation 
research.  In  addition  to  requesting  basic  demographic  information  fi’om  each  student  (name,  age, 
time  in  service,  rank,  time  in  rank,  source  of  rank,  and  branch),  the  Biographical  Information 
Questionnaire  focused  on  seven  different  indicators  of  military  experience: 

•  Previous  duty  positions,  the  units  in  which  the  positions  were  held,  and  the  duration  of  the 
assignments; 

•  Types  of  Army  training  courses  completed; 

•  Courses  completed  during  any  prior  enlisted  service; 

•  Experiences  at  Combat  Training  Centers; 

•  Experiences  (in  combat  or  in  stability  and  support  operations)  diiring  deployments  outside 
the  continental  U.S.; 

•  Experiences  during  platoon,  company,  or  staff  assignments  with  each  of  14  different  key 
military  events;  and 
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Experiences  with  military  training  simulations  and  commercial  computer  games. 


Test  of  General  Cognitive  Ability 

The  Wonderlic  Personnel  Test  was  used  to  obtain  a  measure  of  the  general  cognitive  ability 
of  participants.  The  current  version  of  this  instrument  is  based  on  the  result  of  over  60  years  of 
testing  and  research  by  a  wide  variety  of  organizations  around  the  world  for  many  different  job 
positions  and  for  education  levels  from  tiie  seventh  grade  through  post-college  graduate.  The 
test  consists  of  50  questions  covering  a  wide  variety  of  problem  types.  The  questions  are 
arranged  in  the  test  booklet  in  order  of  their  difficulty,  beginning  at  a  modest  level  and  increasing 
gradually  in  difficulty.  Test  scores  are  based  on  the  number  of  questions  answered  correctly  in  a 
12-minute  test  period. 

Research  has  shown  that  participants  with  higher  scores  tend  to  leam  more  quickly,  master 
more  complex  material,  and  make  better  judgments  when  information  is  lacking  than  those  with 
lower  scores.  Participants  with  lower  scores  tend  to  require  more  time,  specific  instractions,  and 
stmctured  job  routines  than  those  with  higher  scores.  The  median  score  obtained  by  high  school 
and  college  graduates  in  the  U.S.  is  21  and  29,  respectively.  Dodrill  (1983)  reported  the  long¬ 
term  test-retest  reliability  of  the  test  was  .94.  After  finding  a  correlation  of  .91  between  scores 
obtained  on  the  Wonderlic  Personnel  Test  and  the  Wechler  Adult  Intelligent  Scale,  Dodrill  and 
Warner  (1988)  concluded  that  the  Wonderlic  test  is  valuable  as  a  brief  measure  of  general 
intelligence.  Detailed  information  and  sample  questions  for  the  Wonderlic  Personnel  Test  can  be 
obtained  from  the  web  site  of  its  producer,  httD://www.wonderlic.com. 

Decision-Making  Style  Inventory 

We  used  a  method  developed  by  Nygren  (2000)  to  assess  each  student’s  propensity  to  use 
one  or  more  of  three  distinct  styles  or  strategies  for  decision  making.  A  copy  of  the  instrumeiit, 
the  Decision-Making  Style  Inventory,  was  provided  to  the  authors  for  use  in  this  research.  It  is 
given  on  pages  A-4  and  A-5  of  Appendix  A.  The  Decision-Making  Style  Inventory  consists  of 
45  items,  15  on  each  of  three  scales.  The  student  indicated  his  level  of  agreement  or 
disagreement  to  each  item  using  a  six-point  rating  scale,  yielding  a  total  score  between  15  and  90 
on  each  scale.  Scores  on  the  three  scales  were  shown  to  be  highly  correlated  with  the  use  of  an 
Analytical,  an  Intuitive,  ox  ei  Regret-based  Emotional  decision-making  style  in  two  separate 
studies,  each  using  over  800  college  students  (Nygren,  2000).  The  mean  (and  standard 
deviation)  of  scores  reported  from  one  of  these  studies  were,  respectively,  64.8  (9.8)  for  the 
Analytical  scale,  61.3  (9.4)  for  the  Intuitive  scale,  and  58.5  (1 1.6)  for  the  Regret-based 
Emotional  scale. 

Nygren  (2000)  reported  that  scores  on  the  three  scales  were  essentially  orthogonal, 
suggesting  that  the  scales  measure  independent  unipolar  dimensions  and  not  opposite  ends  of  one 
or  more  unidimensional  continua.  Consequently,  a  given  individual  could  score  high  (or  low)  on 
more  than  one  scale.  The  construct  validity  of  the  Decision-Making  Style  Inventory  was 
established  by  relating  scores  among  the  three  scales  to  other  well-established  measures  of 
individual  thinking  and  decision-making  strategies.  In  summary,  these  comparisons  support  the 
following  conclusions: 
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•  High  analytic  individuals  are  more  likely  to  employ  rational-thinking  styles,  be 
performance-  and  training-goal  oriented,  and  have  a  high  need  for  cognition  orientation,  and 
are  less  likely  to  be  risk  seeking,  impulsive,  and  work  intolerant. 

•  High  intuitive  individuals  are  more  likely  to  employ  aq>eriential-thinking  styles,  be  more 
goal  oriented,  risk  seeking  and  impulsive,  show  hi^er  self-esteem  and  have  a  greater  belief 
in  luck,  and  are  less  likely  to  exhibit  self-deception,  depression,  or  causal  uncertainty. 

•  High  regret-based  emotional  individuals  tend  to  score  higher  in  personal  harm  avoidance, 
woridoad  intolerance,  performance-goal  orientation,  self-handicapping,  both  personal  and 
judgmental  self-doubt,  depression,  and  causal  uncertainty,  and  they  tend  to  score  lower  in 
risk  seeking  and  self  esteem. 

The  results  of  several  studies  (Nygren,  2000;  Nygren  &  White,  2002;  White  &  Nygren, 
2002)  have  shown  that  a  propensity  toward  use  of  either  an  analytic  or  intuitive  decision-making 
style  can  affect  performance  on  complex  computer-generated  tasks  that  simulate  those  performed 
by  aircraft  pilots  during  flight.  These  studies  suggest  a  reliance  on  an  analytical  style  may  lead 
to  poorer  performance  than  relying  on  an  intuitive  approach,  particularly  when  there  are  multiple 
subtasks,  each  with  its  own  performance  criteria.  In  addition,  the  studies  suggest  that  as 
workload  levels  increased,  a  more  intuitive  style  led  to  higher  levels  of  performance.  A  regret- 
based  emotional  approach  to  decision  making  did  not  appear  to  affect  performance  on  these 
tasks,  leading  Nygren  (2000)  to  suggest  that  this  decision-making  strategy  might  be  most 
relevant  in  more  risky  and  realistic  decision  environments. 

FSC  Questionnaire 

A  copy  of  the  FSC  Questionnaire  is  given  on  pages  A-6  to  A- 16  of  Appendix  A.  This  six-part 
questionnaire  was  developed  by  the  authors  to  document  the  reactions  of  students  in  the  FSC 
group  to  their  experience  with  FSC.  The  FSC  Questionnaire  asked  each  student  to  indicate: 

•  The  extent  to  which  he  became  personally  involved  or  immersed  in  his  game-playing 
experience; 

•  His  perceptions  of  the  overall  training  value  of  the  FSC  playing  experience; 

•  His  perceptions  of  the  training  value  of  specific  scenarios  he  played  using  FSC; 

•  His  opinions  concerning  the  importance  and  quality  of  the  various  types  of  FSC  fidelity; 

•  His  assessment  of  the  extent  to  which  the  game  permitted  him  to  perform  and  practice  action 
items  that  it  was  designed  to  permit,  as  well  as  his  assessment  of  additional  action  item 
capabilities  that  should  be  built  into  future  versions  of  FSC;  and 

•  His  written  responses  to  a  series  of  open-ended  questions  concerned  with  his  experience  with 
FSC  and  his  opinion  about  the  potential  of  FSC  for  improving  the  decision-making  skills  of 
ICCC  students. 
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The  specific  form  snd  substance  of  items  included  in  the  FSC  Questionnaire  were  determined 
using  inputs  from  and  consultation  with  ICCC  instructors  at  Fort  Benning,  the  FSC  developers, 
and  contracted  military  subject  matter  experts. 

It  must  be  noted  that  only  the  24  items  in  Part  1  of  the  questionnaire  were  derived  from 
methods  used  in  earlier  research.  A  search  of  the  literature  did  not  yield  any  references  that 
described  metiiods  for  measuring  a  player’s  involvement  or  immersion  in  a  computer-based 
game.  However,  references  were  found  for  a  questionnaire  that  assessed  reliably  at  least  some 
aspects  of  the  phenomenon  called  presence  in  research  on  human  behavior  in  a  computer¬ 
generated  virtual  simulation  environment  (Singer  &  Witmer,  1996;  Witmer  &  Singer,  1994, 
1998;  Witmer,  Singer  and  Jerome,  in  press).  The  cited  researchers  showed  that  the  32-item, 
Version  3.0  Presence  Questionnaire  was  a  reliable  measure  of  factors  that  encourage 
involvement  and  enable  immersion  in  a  virtual  environment.  The  authors  selected  and  edited 
many  items  from  that  questionnaire  to  make  them  applicable  to  a  computer-based  game 
environment. 


Janus  Scenario  Assessment  Checklist 

A  detailed  description  of  the  Janus  combat  simulation  system  and  procedures  for  its  use  as 
a  training  platform  are  found  in  Grotte  et  al.  (1995).  Janus  is  a  two-sided,  high  resolution, 
interactive  simulation  of  realistic  battlefield  events.  The  simulation  is  driven  by  a  sof^^e 
system  hosted  on  a  mainframe  computer.  Janus  has  sufficient  resolution  to  model  individual 
fighting  systems  or  individual  Soldiers  and  can  model  realistically  up  to  brigade-size  forces. 
While  originally  developed  to  serve  as  an  analytic  tool  during  the  design  and  development  of 
materiel  systems  and  force  organizations,  Janus  can  also  be  used  as  a  training  simulation  and  as  a 
vehicle  for  assessing  human  performance.^  The  interactive  mode  of  operation  used  during 
human  training  and  assessment  applications  allows  military  commanders  and  staff  to  practice  the 
decision-making  processes  required  to  synchronize  battlefield  systems  over  successive  phases  of 
a  mission.  Janus  displays  digitized  terrain  data  on  computer  monitor  screens  in  a  format  familiar 
to  military  users.  It  models  accurately  a  wide  assortment  of  fiiendly  and  enemy  force  elements 
as  a  function  of  each  fighting  system’s  capabilities,  as  affected  by  factors  such  as  terrain, 
weather,  and  visibility.  Players  of  Janus  must  consider  and  synchronize  all  aspects  force 
employment  just  as  they  would  in  actual  combat.  If  they  neglect  key  considerations,  the 
simulation  will  highlight  the  planning  failure  during  the  battle.  Conversely,  Janus  will  reinforce 
positively  a  fully  integrated  and  synchronized  plan. 

In  this  research,  the  Janus  exercise  was  designed  to  be  a  criterion  test  of  the  training 
effectiveness  of  FSC.  It  was  adnumstered  to  individual  officers  in  both  the  FSC  and  control 
groups.  The  exercise  was  driven  by  a  scenario  developed  to  facilitate  an  assessment  of  each 


^  There  are  several  constraints  for  using  Janus  routinely  as  a  training  device  for  individual  officers.  Its  use  requires 
not  only  a  tnainffame  computer,  but  also  other  multiple  types  of  physical  and  human  resources.  For  example,  four 
personnel,  in  addition  to  an  ARI  researcher,  were  required  to  support  our  use  of  Janus  in  this  evaluation  of  FSC 
training  effectiveness.  Janus  is  used  typically  once  or  twice  during  each  ICCC  class  to  expose  selected  students  to 
the  experience  of  executing  a  simulated  battlefield  mission  scenario.  However,  time  and  other  resource 
requirements  do  not  allow  the  ICCC  to  permit  every  student  in  a  class  to  practice  repeatedly  with  Janus  the  cognitive 
skills  that  were  the  focus  of  our  evaluation. 
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participating  studrat’s  ability  to  plan  and  direct  a  company-level  operation.  It  also  pushed  the 
limits  of  his  adaptive  abilities.  The  scenario  was  based  on  a  tactical  environment  in  which 
uncertain  conditions  emerged.  The  student  was  required  to  adapt  in  order  to  complete  his 
mission  successfully.  To  support  that  scenario,  a  battalion  operations  order  was  developed 
outlining  a  battalion  task  force-level  mission.  In  the  battalion  operations  order,  each  student  was 
tasked  to  plan  and  execute  a  multi-phased  company-level  mission.  The  terrain  for  the  mission 
was  limited  to  the  McKenna  MOUT  Site  and  a  four  grid  square  area  surrounding  the  site.  The 
comply  mission  began  with  a  coordinated  attack  in  sector  to  seize  an  objective,  followed  by  the 
establishment  of  hasty  defensive  positions  and  preparation  for  continuing  the  attack. 

A  Janus  Scenario  Assessment  Checklist  was  developed  by  the  authors  to  assess  each 
participant’s  ability  to  adapt  to  changing  and  uncertain  battlefield  conditions  during  the  Janus 
exercise.  A  copy  of  this  checklist  is  given  on  pages  A- 17  to  A- 19  of  Appendix  A.  The  checklist 
permitted  the  exercise  observer/controller  to  evaluate  each  student’s  ability  to  adapt  to  as  many 
as  23  mission  critical  tasks  required  to  execute  the  mission  successfully.  As  illustrated  in  Table 
1,  the  checklist  was  used  to  indicate  (a)  whether  the  participant  recognized  the  need  to  change 
some  aspect  of  the  operational  plan  due  to  emerging  battlefield  conditions,  (b)  whether  he  was 
promoted  by  the  observer/controller  through  a  series  of  targeted  questions  to  recognize  this 
need,  (c)  the  doctrinally  determined  appropriateness  of  any  change  made  to  the  plan,  and,  if  a 
decision  was  made  to  change  a  plan,  (d)  the  outcome  of  implementing  that  decision.  The 
assessment  checklist  took  into  account  the  possibility  that  an  adaptive  decision  might  not  be 
reqwed  for  a  specific  task  because  the  task  was  accommodated  by  the  initial  plan  or  by  previous 
activities,  or  because  the  exercise  was  terminated  before  the  task  was  required  to  be  performed. 

Two  measures  were  derived  fi-om  the  Janus  Scenario  Assessment  Checklist  to  reflect  the 
performance  of  students  while  they  commanded  their  company  in  the  simulated  mission,  hi 
consultation  with  the  ICCC  small  group  instructors,  the  ten  nominal  values  derived  fi’om  the 
checklist  were  Judged  to  reflect  an  ordinal  scale  of  the  adaptability  of  the  student’s  decision¬ 
making  performance  for  each  of  the  23  tasks.  The  highest  level  of  adaptability  was  given  a  scale 
value  of  1  and  the  the  lowest  level  of  adaptability  was  assigned  a  scale  value  of  1 0.  Task 
performance  was  judged  to  be  adaptive  if  it  obtained  a  scale  value  of  6  or  less.  The  percentage 
of  tasks  to  which  an  adaptive  was  made  was  determined  for  each  Janus  player.  The  second 
measure  derived  fi-om  the  checklist  was  based  on  whether  the  observer/controller  provided  a 
prompt  to  encourage  a  non-responsive  participant  to  make  an  adaptive  response.  The  percentage 
of  tasks  for  which  a  prompt  was  provided  was  determined  for  each  student. 

In  addition  to  assessing  the  performance  of  students  with  the  checklist,  the 
observer/controller  identified  whether  a  Janus  player  had  committed  any  of  ten  common 
doctrinal  errors  during  his  execution  of  the  mission  (see  page  A-20  of  Appendix  A).  This 
determination  was  made  after  the  conclusion  of  a  Janus  session  based  on  the 
observer/controller’s  review  of  notes  he  had  taken  on  the  flow  of  the  battle  that  had  just 
occurred.  Consequently,  a  third  measure  of  performance  during  the  Janus  exercise  was  the 
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In  addition  to  Iwing  a  data  collection  opportunity,  the  Janus  exercise  was  a  learning  experience  for  the  participants. 
As  such,  instructional  steps  were  taken  by  the  observer/controller  to  insure  that  students’  plans  and  actions  would 
facilitate  some  level  of  learning  and  mission  success. 
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number  of  doctrinal  errors  each  Janus  player  committed  during  his  command  of  his  simulated 
company. 


Table  1 


Attributes  of  Adaptive  Decisions  Captured  by  the  Janus  Scenario 
Assessment  Checklist  _ 


Decision 

Category 

Recognized 
Need  to 
Change? 

Prompt 

Provided? 

Implemente 

dGood 

Decision? 

Decision 

Outcome 

1 

Yes 

No 

Yes 

Positive 

2 

No 

Yes 

Yes 

Positive 

3 

Yes 

No 

No 

Positive 

4 

No 

Yes 

No 

Positive 

5 

Yes 

No 

No 

Negative 

6 

No 

Yes 

No 

Negative 

7 

No 

No 

Did  nothing 

8 

No 

Yes 

Did  nothing 

9 

Yes 

No 

Did  not  deviate  fix)m  plan 

10 

Yes 

Yes 

Did  not  deviate  fi'om  plan 

Plan 

Plan  accommodated  activity 

Other 

No  adaptive  thought  required  because  previous 
linking  task  was  performed  correctly 

NC 

Not  Completed  -  The  task  was  not  required  because 
the  mission  was  terminated  before  its  scheduled 
occurrence 

Evaluation  Design  and  Procedures 

Figure  1  summarizes  the  timeline  and  procedures  for  data  collection.  During  the  first  week 
of  each  class,  pre-FSC  measures  were  obtained  of  the  students’  status  and  experience  in  military 
service,  their  general  cognitive  ability,  and  their  decision-making  style.  The  conduct  of  the  FSC 
group  seminars  and  the  behavior  of  students  in  fiiese  seminars  were  monitored  closely  and 
recorded  for  FSC  train-up  and  play.  The  FSC  Questionnaire  was  administered  to  all  students  in 
the  FSC  group  following  their  last  scheduled  exposure  to  the  game.  Because  FSC  was 
experienced  over  successive  periods  by  small  groups  of  students,  the  FSC  Questionnaire  was 
likewise  administered  to  small  groups  of  students  over  a  two-three  day  period.  On  the  day 
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followdng  FSC  play,  all  students  in  both  the  FSC  and  control  groups  were  re-administered  the 
decision-making  style  inventory  to  obtain  one  of  the  post-FSC  evaluation  measures.  Over  the 
next  nine  sessions  of  the  ICCC,  Janus  data  were  collected  from  students  in  the  FSC  and  control 
groups.  Descriptions  of  the  procedures  employed  for  conducting  and  collecting  data  during  FSC 
and  Janus  sessions  are  presented  in  the  next  two  sections  of  the  report. 


Figure  1.  A  summary  of  the  timeline  and  procedures  for  the  FSC  evaluation. 


Conduct  and  Observations  of  FSC  Play 

FSC  was  programmed  to  permit  game  play  in  any  one  of  three  different  scenarios: 
reconnaissance,  breach,  and  cordon  and  search.  During  early  stages  of  planning  for  this 
evaluation,  the  authors  and  their  colleagues  met  with  the  small  group  instructors  from  the  ICCC 
to  discuss  methods  and  procedures  for  inserting  FSC  into  the  program  of  instruction  and  for 
assessing  the  students  performance  as  they  played  the  game.  Initial  plans  determined  that  six 
computers  would  be  set  up  to  play  FSC.  Each  participant  was  to  play  the  game  for 
approximately  20  hours  during  the  period  of  ICCC  instruction  for  operations  in  urban 
enviro^ents.  Furthermore,  it  was  anticipated  that  small  group  instructors  would  use  a  series  of 
checklists  to  evaluate  how  well  the  students  planned  and  executed  their  missions,  and  how  well 
they  adapted  to  emerging  threats  and  conditions. 
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However,  the  realities  of  conducting  field  research  overcame  tiie  best  of  scientifically  valid 
intentions.  For  one  thing,  the  number  of  computers  that  were  available  and  set  up  to  play  FSC 
for  any  given  session  varied  fi'om  two  to  five.  Furthermore,  while  the  small  group  instructors 
took  part  in  an  initial  training  session,  they  were  unable  to  implement  any  formal  evaluation 
procedures  during  FSC  play.  They  had  to  keep  the  conduct  of  their  seminars  on  the  same 
schedule  as  other  seminars,  and  they  had  to  continue  participating  in  the  instruction  of  students 
who  were  not  playing  FSC.  They  were  also  unable  to  maintain  a  fixed  schedule  for  students  to 
play  FSC  and  could  not  ensure  that  students  played  each  of  the  three  different  scenarios.  In 
short,  there  was  no  formal  assessment  of  performance  as  students  played  FSC  and  no  fixed 
procedures  for  playing  FSC. 

One  researcher  observed  the  behavior  of  all  students  during  the  FSC  train-up  and 
subsequent  play.  Records  were  made  of  the  context  and  the  conduct  of  the  FSC  playing  sessions. 
The  time  each  student  began  and  ended  each  experience  with  FSC  was  recorded.  It  was  noted 
when  a  player  modified  his  plans  by  creating  and  implementing  a  fragmentary  order  during 
mission  execution.  Because  the  recorded  observations  of  this  researcher  are  the  only  description 
of  the  procedures  used  during  FSC  play,  they  are  summarized  in  the  next  two  paragraphs. 

All  students  in  the  FSC  group  were  given  the  FSC  User  Manual  prior  to  their  participation 
in  the  initial  FSC  demonstration  and  train-up  session.  During  this  session,  students  became 
familiar  with  the  FSC  user  interface  and  with  procedures  for  controlling  game  functions. 
Subsequent  FSC  playing  sessions  consisted  typically  of  individual  students  working  through  a 
scenario  within  a  fixed  time  (usually  about  60  minutes).  On  other  occasions,  students  were 
formed  into  small  groups  to  work  fiirough  a  scenario,  with  one  student  controlling  computer 
fimctions  and  others  sharing  in  the  planning  and  execution  of  the  mission.  On  one  occasion,  a 
small  group  instructor  formed  two  groups  of  seven  students  each  and  instructed  them  to  complete 
a  scenario.  Upon  completion  of  the  scenario,  the  instructor  determined  which  group  had 
conducted  the  best  mission.  This  instructor  then  conducted  a  brief  after  action  review  based  on 
the  plan  and  execution  of  the  better  mission. 

One  small  group  instructor  specified  which  scenario  was  to  be  played  by  students  in  his 
seminar  during  each  FSC  session.  The  other  small  group  instructor  allowed  students  in  his 
seminar  to  play  any  scenario  they  wanted.  Except  for  the  one  exercise  described  in  the  preceding 
paragraph,  the  small  group  instructors  were  unable  to  participate  with  the  students  in  after  action 
reviews  of  their  FSC  play.  Further,  there  were  never  any  attempts  by  the  students  to  utilize  the 
capabilities  of  FSC  that  support  an  after  action  review. 

Conduct  and  Observations  of  Janus  Play 

The  use  of  Janus  to  obtain  criterion  measures  of  FSC  training  effectiveness  began  after 
students  in  fiie  FSC  group  completed  playing  the  game,  and  after  students  in  both  groups 
completed  the  ICCC  instruction  on  li^t  Infantry  company  offensive  operations  in  an  urban 
environment.  On  each  of  nine  days,  one  student  fi-om  bolh  the  FSC  and  the  control  groups 
completed  a  tactical  exercise  using  the  Janus  simulation.  Due  to  the  limited  availability  of  Janus 
resources  only  36  of  the  54  students  (18  from  each  group)  participated  in  the  Janus  exercise. 

Two  steps  were  taken  to  accommodate  this  constraint.  First,  small  group  instructors  fi)r  the  FSC 
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group  selected  students  they  thought  would  be  least  negatively  impacted  by  missing  regularly 
scheduled  classroom  instruction  and  who  would  most  benefit  from  participating  in  the  Janus 
exercise.  Second,  the  authors  worked  in  conjunction  with  small  group  instructors  for  frie  control 
group  to  select  control  students  who  would  be  most  similar  to  those  from  the  FSC  group  in  terms 
of  general  cognitive  ability,  decision-making  style,  and  relevant  indicators  of  military 
experience.  One  student  from  one  group  was  scheduled  each  day  for  a  morning  session  and  one 
from  the  other  group  for  an  afternoon  session.  The  assignment  to  a  morning  or  to  afternoon 
session  alternated  between  the  two  groups.  The  scheduling  of  students  for  the  Janus  exercise 
was  counterbalanced  over  successive  days  to  control  for  possible  confounding  effects  of  general 
cognitive  ability. 

All  students  selected  to  participate  in  the  Janus  exercise  received  a  Janus  exercise  battalion 
operations  order  at  least  one  day  before  their  scheduled  session.  Upon  receiving  the  operations 
order,  students  were  instructed  to  read  and  analyze  the  order,  develop  an  operations  plan  for  their 
company,  and  be  prepared  to  communicate  their  task  organization  and  their  concept  of  the 
operation.  They  were  not  required  to  utilize  the  implements  that  were  used  typically  for  briefing 
a  plan  dtiring  ICCC  instruction  (e.g.,  transparencies,  hand-drawn  terrain  maps,  battle  boards, 
etc.). 

Upon  arriving  for  the  exercise,  each  student  was  asked  to  commimicate  his  task  organization 
and  his  concept  of  the  operation  to  the  observer/controller.  If  his  observer/controller  detected  a 
serious  flaw  in  the  student’s  plan,  he  reviewed  relevant  portions  of  the  company  operations  order 
with  the  student  and  prompted  him  to  make  changes.  Because  of  these  prompts,  most  students 
recognized  the  need  to  change  their  plans  and  responded  by  making  appropriate  changes. 
Students  who  elected  not  to  make  changes  were  allowed  to  continue  the  exercise  with  their 
original  plan.^ 

After  discussing  his  company  operations  order  with  the  observer/controller,  the  student 
worked  with  a  Janus  programmer  who  inputted  the  student’s  task  organization  and  movement 
routes  into  the  Janus  computer  system.  Hence,  while  students  had  control  over  the  placement 
and  movement  routes  of  all  maneuver  elements,  they  did  not  have  to  actually  control  this  aspect 
of  Janus  play.  Instead,  they  could  devote  their  attention  to  commanding  and  controlling  their 
force  elements  during  both  the  initial  setup  of  these  elements  and  any  subsequent  changes  that 
they  implemented  to  the  original  operations  order.  The  input  of  the  initially  planned  blue  force 
play  took  from  forty  minutes  to  one  and  one-half  hours. 


Before  beginning  mission  execution,  company  assets  and  call  signs  were  described  to  the 
student.  For  communication  purposes,  a  subject  matter  expert  assumed  the  roles  of  all  company 
entities  subordinate  to  the  company  commander.  The  student  used  verbal  commands  to 
communicate  orders  to  the  subject  matter  expert,  replicating  call  signs  and  instructions  as  if 
actual  radio  communications  were  in  effect.  The  subject  matter  expert  received  the  verbal 
commands,  responded  verbally  to  established  cues,  and  initiated  situation  reports  in  the  same 


^  Throughout  the  Janus  exercise,  prompting  by  the  observe/controller  was  used  when  required  to  insure  that  the 
exercise  was  a  training  event. 
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manner.  No  radio  equipment  was  used  to  commumcate  during  the  exercise.  Mission  execution 
began  when  the  student  indicated  that  his  operations  order  had  been  correctly  entered  into  the 
Janus  software  and  ftiat  he  understood  the  administrative  instructions. 

During  mission  execution,  the  student,  the  observer/controller,  and  the  blue  force  Janus 
programmer  viewed  a  computer  monitor  screen  that  displayed  only  blue  force  elements  and  their 
intended  routes  of  maneuver  on  the  designated  battlefield  terrain.  On  the  other  side  of  a 
partition,  a  subject  matter  expert,  serving  as  a  role  player,  and  the  opposing  force  Janus 
programmer  viewed  another  computer  screen  that  displayed  only  the  opposing  force  elements. 
The  observer/controller  and  the  subject  matter  expert  initiated  a  series  of  events  (cues)  to  which 
the  student  could  respond.  All  of  these  events  were  identified  through  situation  reports  given  by 
company  subordinates  to  the  student.  Upon  receiving  a  report,  the  student  had  the  opportunity  to 
analyse  a  potential  threat  and  modify  his  plan  accordingly.  The  observer/controller  evaluated  the 
student’s  response  to  each  event  using  the  Janus  Scenario  Assessment  Checklist.  If  the 
observer/controller  suspected  ftiat  the  student  did  not  perceive  correctly  the  need  to  modify  his 
plan  after  receiving  a  situation  report,  the  observer/controller  gave  a  verbal  prompt.  For 
example,  if  a  notional  scout  platoon  leader  told  the  student  that  a  mechanized  enemy  force  had 
been  spotted  moving  toward  a  key  objective  area,  the  student  would  be  required  to  respond  in 
order  to  defend  or  secure  that  objective.  If  the  student  neglected  to  respond  appropriately,  the 
observer/controller  would  ask  questions  designed  to  prompt  an  appropriately  adaptive  response. 
The  following  questions  are  examples  of  these  prompts:  Do  you  understand  that  those  enemy 
mechanized  forces  are  moving  at  14  mph?  Do  you  realize  that  your  dismounted  troops  are 
traveling  at  4  mph?  Will  the  enemy  forces  arrive  at  that  objective  before  your  forces  do?  How 
will  tiiat  affect  your  forces  in  the  area?  How  will  that  affect  your  overall  mission?  After  the 
prompts,  the  student  was  free  to  choose  a  new  course  of  action  or  to  maintain  his  current  one. 
The  observer/controller  rated  each  response  accordingly. 

The  observer/controller  ended  the  exercise  about  thirty  minutes  before  the  end  of  the 
scheduled  four-hour  session  or  when  the  simulated  battle  was  over  (i.e.,  there  was  a  near 
destruction  of  friendly  or  enemy  assets,  whichever  came  first).  After  the  student  completed  a 
five-minute  break,  the  observer/controller  conducted  an  after  action  review  with  the  student. 
Critical  events,  decisions,  and  actions  were  addressed  while  the  student  and  observer/controller 
observed  a  hi^-speed  replay  of  the  mission  execution.  The  observer/controller  pointed  out 
strengths  and  weaknesses  of  the  student’s  plan  and  execution,  asked  doctrinally  relevant 
questions,  and  allowed  the  student  to  respond.  Upon  completion  of  the  after  action  review,  the 
student  was  asked  not  to  discuss  his  experience  with  his  fellow  students.  He  was  then  fi'ee  to 
leave.  The  observer/controller  then  completed  his  evaluation  of  the  student’s  performance  and 
wrote  a  brief  summary  of  the  exercise. 


RESULTS 

The  design  of  this  evaluation  research  called  for  three  sets  of  analyses:  (a)  comparisons  of 
measures  obtained  fi-om  officers  in  the  FSC  and  control  groups,  (b)  an  examination  of 
observations  during  FSC  play  and  ratings  by  students  who  played  the  game,  and  (c)  a  study  of 
data  pooled  over  students  in  both  groups.  The  results  of  these  analyses  are  presented  in  the 
following  three  major  sections.  All  inferential  test  statistics  presented  were  determined  to  be 
statistically  significant  when  their  probability  of  occurrence  was  less  than  or  equal  to  0.05. 
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Comparison  of  FSC  Group  and  Tontrol  Groun 

Except  for  three  aspects  of  the  their  military  experience  (described  in  the  second  paragraph 
below),  students  in  the  FSC  and  control  groups  were  not  different  with  respect  to  either  the  pre- 
FSC  measures  (i.e.,  the  biographical  questionnaire,  the  test  of  general  cognitive  ability,  and  the 
decision-making  style  inventory)  or  the  post-FSC  measures  (i.e.,  the  decision-making  style 

inventory  and  performance  during  the  Janus  exercise).  These  results  are  described  in  subsequent 
labeled  sections. 


Biographical  Information 

Distributed  about  equally  across  the  two  groups,  the  mean  (and  standard  deviation)  age  of 
the  students  was  29.6  years  (3.7)  and  their  time  in  service  was  8.6  years  (4.5).  Thirty-two 
students  (59.3%)  were  Army  Infantrymen,  seven  (12.9%)  were  from  other  branches  of  the  Army, 
tiiree  (5.6%)  were  from  the  U.S.  Marine  Corps,  and  the  remaining  twelve  (22.2%)  were 
international  students  representing  twelve  different  foreign  military  services.  Thirty-two  (59.3%) 
students  were  captains  and,  except  for  one  major  (from  a  foreign  service),  the  others  held  the 
ra^  of  first  lieutenant.  Twenty-nine  of  the  54  students  (55.8%)  indicated  they  had  experience 
with  military  training  simulations  and  22  (43.1%)  indicated  they  played  commercial  computer 
games.  Of  the  22  students  who  played  computer  games,  only  four  indicated  they  played  games 
10  houre  or  more  per  week.  The  mean  (and  standard  deviation)  number  of  hours  per  week  the 
remaining  18  students  reported  they  played  computer  games  was  3.6  hours  (1.8). 

Nineteen  indices  reflecting  the  type  and  level  of  each  student’s  military  experience  were 
developed  from  other  responses  obtained  with  the  Biographical  Information  Questionnaire. 

These  indices  and  the  methods  used  to  generate  them  are  given  in  Appendix  Table  B-I.  There 
was  a  significant  difference  between  the  FSC  and  control  groups  for  only  three  of  these  indices 
(see  Table  2).  The  students  in  the  FSC  group  reported  that  a  higher  mean  percentage  of  their 
(non-student)  duty  time  was  spent  in  company-level  assignments  than  did  students  in  the  control 
group,  F(l,  50)  =  3.9.  On  the  other  hand,  a  higher  percentage  of  students  in  the  control  group 
than  in  the  FSC  group  reported  that  they  were  prior  enlisted,  F(1 , 50)  =  5. 1 5,  and  had  served  in 
military  operations  in  a  foreign  country,  F(l,  51)  =  4.70. 

Table  2 


Military  Experience  by  Group 


Group 

%  Duty  Time  in 
Company  Echelon 

%  of  Group  with 
Prior  Enlisted 
Service 

%  of  Group  with 
Assignments 
Outside  the  U.S. 

FSC 

25.6(17.4) 

29.6  (4.6) 

44.4  (5.1) 

Control 

16.0(17.4) 

60.0  (5.0) 

73.1  (4.5) 

Note:  Data  shown  are  means  (and  standard  deviations). 


The  difference  between  groups  for  one  other  measure  of  military  experience  approached 
statistical  sigmficance  (p  <  .07)  but  is  worthy  of  description  because  it  is  related  to  one  of  the 
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differences  shown  in  Table  2,  The  students  in  the  control  group  reported  that  a  higher  mean 
percentage  of  their  (non-student)  duty  time  was  spent  in  assignments  below  company  echelon 
level  than  did  those  in  the  FSC  group.  The  means  (and  standard  deviations)  of  this  measure  for 
the  control  and  FSC  groups  were  64.6  percent  (28.5)  and  52.1  percent  (19.6),  respectively. 

General  Cognitive  Ability  and  Decision-Making  Style 

There  was  no  difference  between  groups  for  measures  of  general  cognitive  ability  or 
decision-making  style.  Across  the  two  groups,  the  mean  (and  standard  deviation)  score  on  the 
Wonderlic  test  of  general  cognitive  ability  was  23.7  (5.4).  The  mean  (and  standard  deviation) 
scale  values  for  the  pre-FSC  decision-making  dimensions  of  analytic,  intuitive,  and  regret-based 
emotional  styles  were  72.8  (7.7),  64.5  (8.5),  and  45.9  (11.1),  respectively.  The  corresponding 
values  for  post-FSC  decision-making  style  dimensions  were  72.5  (8.5),  66.1  (9.0),  and  43.0 
(10.1). 


Measures  Derived  From  the  Janus  Exercise 

The  time  that  elapsed  between  the  start  and  end  of  the  mission  execution  phase  of  the  Janus 
exercise  was  recorded  for  each  student.  The  mean  (and  standard  deviation)  mission  time  was 
133.1  minutes  (38.0).  This  time  did  not  differ  for  the  FSC  and  control  groups. 

Analyses  of  the  three  measures  of  student  performance  during  the  Janus  exercise  failed  to 
find  any  difference  between  the  FSC  group  and  control  group.  The  grand  means  (and  standard 
deviations)  over  the  23  Janus  tasks  were  80.2  percent  (18.2)  adaptive  responses  and  14.9  percent 
(9.9)  prompts.  Over  both  groups,  the  mean  (and  standard  deviation)  number  of  doctrinal  errors 
committed  was  2.2  (2.1). 

Observations  of  Students  and  Their  Reactions  to  FSC 


Behavioral  Observations  During  FSC  Demonstration,  Train-Up,  and  Play 

One  set  of  observations  provided  a  basis  for  quantitative  analysis  of  the  time  each  student 
was  involved  with  FSC.  Table  3  shows  the  amount  of  time  students  were  exposed  to  FSC, 
separately  for  sessions  devoted  to  demonstrating  and  preparing  the  students  to  play  FSC  and 
sessions  in  which  they  actually  played  the  game.  In  the  latter  case,  a  distinction  was  made 
between  instances  in  which  the  student  was  the  actual  hands-on  player  of  the  game  and  Aose  in 
which  his  role  was  that  of  an  observer  (either  active  or  inactive)  of  game  playing  activities. 
During  the  demonstration  and  train-up  sessions,  students  were  principally  observers.  The  data 
presented  in  Table  3  indicate  the  typical  student  spent  two  hours  performing  hands-on  tasks 
required  both  to  plan  the  mission  and  to  respond  to  events  that  occurred  during  the  mission 
execution.  In  addition,  he  spent  about  three  hours  observing  others  play  the  game. 
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Table  3 


Student  Exposure  to  FSC  (in  minutes) 


Demonstration 
and  Train-up 

Hands-on  FSC 
Play 

Observational 
FSC  Play 

288  (73) 

120  (95) 

168(81) 

Note:  Data  shown  are  means  (and  standard  deviations). 


In  addition,  it  was  noted  that  many  students  used  major  portions  of  the  time  available  for 
playing  an  FSC  scenario  to  analyze  the  battalion  operations  order  and  to  develop  their  company 
operations  order.  Some  students  spent  considerable  amounts  of  FSC  playing  time  consulting  the 
User  Mantial.  In  these  cases,  there  was  little  time  left  in  the  session  to  complete  the  mission  and 
to  respond  to  unanticipated  events  during  mission  execution. 


Responses  of  FSC  Participants  on  the  FSC  Questionnaire 

Responses  to  queries  in  the  FSC  Questionnaire  by  26  members  of  the  FSC  group  were 
analyzed  separately  for  each  part  of  the  questionnaire.  One  member  of  this  group  did  not 
complete  the  questionnaire.  Separate  analyses  also  were  performed  to  examine  Ae  relationship 
among  results  obtained  fi-om  different  parts  of  the  FSC  Questionnaire  and  the  relationship 
between  results  obtained  fi’om  the  FSC  and  Biographical  Information  Questionnaires.  The 
findings  from  these  analyses  are  summarized  in  subsequent  labeled  paragraphs. 

Involvement  in  FSC  simulated  mission  environment.  Details  of  the  results  obtained  with 
this  part  of  the  questionnaire  are  in  Appendix  Table  B-2.  Responses  to  the  seven-point  rating 
scale  items  for  involvement  indicated  that  students  using  the  FSC  game  became  personally 
involved  in  the  simulated  command  and  control  environment.  Across  24  involvement  items,  the 
mean  (and  standard  deviation)  rating  scale  value  was  4.6  (0.6).  Overall,  57.1  percent  of  the 
ratings  were  restricted  to  the  three  most  positive  values.  In  contrast,  1 8.6  percent  of  the  ratings 
utilized  the  three  most  negative  values  and  24.3  percent  used  the  middle,  neutral  rating  scale 
category.  Looked  at  in  another  way,  50  percent  or  more  of  the  students  gave  positive  responses 
for  17  of  the  24  involvement  questionnaire  items,  a  neutral  response  for  six  items,  and  a  negative 
response  for  only  one  item. 

The  extent  of  their  reported  involvement  was  most  telling  in  terms  of  how  quickly  they 
adjusted  to  the  experiences  provided  by  FSC  (Item  15,  mean  scale  value  =  5.3);  how  they  were 
able  to  control  events  during  the  planning  phase  of  the  mission  (Item  1 ,  mean  scale  value  =  5.5); 
and  how  their  experiences  in  the  FSC  environment  were  affected  minimally  by  extraneous  events 
occurring  in  or  outside  the  classroom  (Item  20,  mean  scale  value  =  5.6).  Students  also  indicated 
they  had  a  compelling  sense  of  moving  around  inside  the  FSC  environment  and  that  there  were 
moments  when  they  were  completely  focused  on  the  task  of  commanding  the  simulated  company 
mission  (Items  1 1  and  23,  respectively,  mean  scale  value  =  5.0  in  both  cases).  The  only  caveat  to 
the  perception  of  being  generally  involved  in  the  FSC  experience  was  the  one  item  receiving  a 
negative  response  (Item  21,  mean  scale  value  =  3.0).  This  response  indicated  that  most  students 
felt  they  had  to  focus  too  much  on  using  the  display  and  control  devices  associated  with  the 
simulation,  at  the  expense  of  the  command  experiences  being  created  by  the  simulation. 


16 


Overall  training  value  ofFSC.  Details  of  the  results  obtained  for  ratings  of  overall 
training  value  ofFSC  are  in  Appendix  Table  B-3.  Student  responses  to  the  ten  items  in  this  part 
of  the  questionnaire  indicated  that  they  believed  using  the  FSC  simulation  was  a  valuable 
learning  experience.  Across  all  ten  items,  60.4  percent  of  the  student  ratings  were  restricted  to 
the  three  most  positive  values  on  the  seven-point  rating  scale.  In  contrast,  18.5  percent  of  the 
ratings  utilized  the  three  most  negative  scale  values  and  21.2  percent  used  the  middle,  neutral 
rating  scale  category.  In  other  words,  the  majority  of  the  students  gave  positive  responses  to 
seven  of  the  overall  training  value  items  in  the  questionnaire  and  a  neutral  response  to  the 
remaining  three  items. 

The  questionnaire  items  receiving  the  highest  mean  rating  indicated  that  the  students 
believed  it  was  desirable  to  use  a  simulation  capability  like  FSC  to  experience  the  possible 
consequences  of  executing  a  tactical  plan  (Item  10,  mean  scale  value  =  5.3),  to  obtain  an 
opportunity  to  practice  their  decision-making  skills  (Item  2,  mean  scale  value  =  5.2),  and  to  have 
a  valuable  learning  experience  (Item  1,  mean  scale  value  =  5.0).  Students  also  indicated  they 
were  given  too  little  time  to  use  FSC  (Item  9,  mean  scale  value  =  4.9). 

Ratings  of  involvement  in  the  FSC  experience  and  ratings  of  overall  training  value  ofFSC 
were  shown  to  be  highly  related  (see  Figure  2).  The  Spearman  rho  statistic  yielded  a  correlation 
coefficient  of  .823  for  this  association.  For  both  statistical  and  practical  reasons,  we  concluded 
that  a  student’s  level  of  personal  involvement  in  his  FSC  experience  was  positively  related  to  his 
perception  of  the  overall  value  of  that  training  experience,  and  vice  versa. 


Mean  Rating  of  Involvement  in  FSC 

Figure  2.  A  scatterplot  showing  the  association  between 
ratings  of  involvement  with  and  overall  training  value  ofFSC. 
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Trainine  value  of  specific  FSC  scenarios.  Details  of  the  results  obtained  for  ratings  of  the 
training  value  of  specific  FSC  scenarios  are  in  Appendix  Table  B-4.  On  the  average,  73.6 
percent  of  the  students  who  played  a  scenario  reported  that  it  was  challenging.  The  percentages 
of  students  reporting  they  had  learned  something  new  were  35.0  percent  for  the  20  students  who 
played  the  reconnaissance  scenario,  45.8  percent  for  the  24  who  played  the  breach  scenario,  and 
63.6  percent  for  the  1 1  who  played  the  cordon  and  search  scenario.  Of  21  students  responding  to 
a  follow  up  question,  38.1  percent  indicated  the  learning  unique  to  the  FSC  experience  was  more 
significant  for  one  scenario  than  for  the  other  two. 

More  detailed  examination  of  the  data  from  this  part  of  the  questionnaire  showed  that 
students  reported  having  different  levels  of  experience  with  the  FSC  overall  and  with  each  of 
three  FSC  scenarios.  The  number  of  times  a  student  reported  he  played  any  of  the  three  FSC 
scenarios  varied  from  2  to  6,  and  most  did  not  experience  all  three  scenarios.  Four  students 
played  only  the  breach  scenario.  Fifteen  played  two  of  the  three  scenarios  (eleven  the 
reconnaissance  and  breach  scenarios,  two  the  cordon  and  search  scenario  and  the  reconnaissance 
scenario,  and  two  the  cordon  and  search  and  the  breach  scenarios).  Seven  participants  reported 
they  played  all  three  scenarios. 

It  was  shown,  however,  that  when  the  students  did  play  a  scenario,  they  tended  to  modify 
their  plans  for  executing  the  mission  in  response  to  emerging  battlefield  conditions.  Across  all 
three  scenarios,  participants  reported  they  made  an  average  of  2.6  modifications  to  their  initial 
operational  plan  when  they  last  played  a  scenario.  However,  the  number  of  reported 
modifications  made  the  last  time  each  student  played  a  scenario  varied  from  none  to  ten  (see 
Table  4). 


Table  4 


Number  of  Modifications  Made  the 
Last  Time  a  Scenario  was  Played 


Number  of 
Modifications 

Number 
(Percentage) 
of  Students 

0 

5  (19.2) 

1 

4(15.4) 

2 

5  (19.2) 

3 

3(11.5) 

4 

5  (19.2) 

5 

2  (7.7) 

6 

1  (3.8) 

10 

_ 1  (3-8) 

Note.  The  numbers  of  modifications  were  summed 
over  the  three  scenarios  for  each  of  the  26  students. 


The  ratings  obtained  for  the  training  value  of  specific  FSC  scenarios  were  examined  further 
for  potential  relationships  with  ratings  obtained  for  involvement  in  and  overall  training  value  of 
FSC.  Three  significant  and  meaningful  relationships  were  found  among  these  data.  Students 
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indicating  they  learned  something  new  about  mission  execution  with  the  breach  scenario  had 
higher  mean  ratings  for  involvement  with  FSC,  /(22)  =  3.95,  and  for  overall  training  value  of 
FSC,  t(22)  =  3.90,  than  those  indicating  they  did  not  learn  anything  new  from  the  breach 
scenario.  The  mean  (and  standard  deviation)  rating  of  involvement  for  these  two  categories  of 
participants  were  5.0  (.53)  and  4.2  (.45),  respectively.  The  mean  (and  standard  deviation)  rating 
of  overall  training  value  were  5.2  (.50)  and  4.1  (.87),  respectively.  Further,  students  indicating 
the  learning  unique  to  FSC  was  more  significant  for  one  scenario  than  the  others  had  higher 
mean  ratings  for  overall  training  value  than  those  indicating  the  contrary,  /(19)  =  2.62.  The  mean 
(and  standard  deviation)  training  value  rating  for  those  indicating  learning  was  more  significant 
for  one  scenario  than  it  was  for  the  others  was  5.4  (.53),  and  for  those  indicating  the  contrary  it 
was  4.4  (.93). 

Each  student  was  encouraged  in  the  FSC  Questionnaire  to  provide  written  comments  in 
response  to  two  requests  for  additional  information.  If  the  student  indicated  he  had  learned 
something  about  mission  execution  by  playing  the  FSC  scenarios  that  had  not  been  learned 
otherwise,  he  was  asked  to  indicate  what  was  learned.  Sixteen  students  provided  comments  to 
this  request;  the  comments  often  students  had  some  common  themes.  Three  indicated  that  FSC 
taiigbt  them  synchronization  of  battlefield  assets  and  another  three  indicated  FSC  taught  them 
hasty  decision-making.  Four  additional  students  indicated  that  using  FSC  taught  them  aspects  of 
maneuver  element  movement  and  situational  awareness  that  they  had  not  learned  elsewhere  in 
the  course. 

The  second  request  was  directed  at  the  question  of  whether  learning  unique  to  FSC  about 
mission  execution  was  more  significant  for  one  scenario  than  for  the  others.  Eleven  students 
responded  to  this  question;  the  comments  of  six  students  had  some  common  themes.  Three 
remarked  tiiat  synchronization  was  more  complex  for  the  breach  scenario  than  for  other 
scenarios.  Three  others  indicated  that  they  learned  more  when  scenarios  were  relatively  difficult. 

FSC  fidelity.  Details  of  the  opinions  expressed  about  the  fidelity  of  FSC  portrayal  of 
simulated  events  are  in  Appendix  Table  B-5.  Most  students  (70.6%)  indicated  all  four  types  of 
fidelity  for  FSC  visual  displays  were  important.  Most  students  (88.0%)  also  indicated  that  the 
fidelity  of  the  FSC  displays  was  adequate  to  excellent,  though  the  mean  rating  on  a  5-point  scale 
for  the  goodness  of  blue  force  fidelity  (3.1)  was  lower  than  it  was  for  enemy  force,  physical,  or 
psychological  fidelity  (3.5, 3.6,  and  3.4,  respectively).  When  asked  to  rank  order  the  relative 
importance  of  these  four  types  of  fidelity,  most  participants  (87.5%)  gave  blue  force  fidelity  the 
highest  two  ranks  of  1  or  2.  The  mean  rank  assigned  to  blue  force  fidelity  was  1 .2,  while  that  for 
the  other  three  was  2.7, 2.9,  and  2.8). 

The  data  obtained  for  the  ratings  of  FSC  fidelity  were  examined  further  for  their 
relationship  with  ratings  for  involvement  in  and  overall  training  value  of  FSC.  Four  meaningful 
and  significant  relationships  were  identified.  Ratings  of  the  goodness  of  the  blue  force  tactiral 
fidelity  were  positively  related  to  involvement  in  FSC  (Spearman’s  rho  =  .503)  and  to  perceived 
training  value  of  the  game  {rho  =  .683).  Ratings  of  psychological  fidelity  also  were  positively 
related  to  involvement  (Spearman’s  rho  -  .437)  and  to  training  value  {rho  =  .654). 
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Each  student  was  encouraged  in  this  part  of  the  FSC  Questionnaire  to  provide  written 
comments  that  described  FSC  features  that  supported  his  assessment  of  the  importance  or  quality 
of  each  of  the  four  types  of  fidelity.  Fifteen  to  nineteen  students  provided  written  comments  to 
these  four  queries.  The  content  of  these  comments  was  quite  varied  and  seemed  to  be  driven  by 
the  wording  of  the  definitions  provided  for  each  type  of  fidelity  in  the  FSC  Questionnaire.  While 
seven  or  more  students  indicated  that  the  physical  environment  and  the  enemy  force  activities 
were  realistic,  up  to  nine  wrote  comments  indicating  improvements  were  needed  in  the  portrayal 
of  blue  force  movement  and  the  responsiveness  of  blue  force  to  commands  given  in  an 
operations  or  fragmentary  order.  At  least  three  students  indicated  that  the  following  enemy  force 
events  and  actions  were  unrealistic:  They  always  fight  to  the  death,  they  did  not  respond  to  blue 
force  troops  within  line  of  sight,  and  they  were  not  challenging  enough  or  their  formations  were 
too  basic.  The  issue  concerning  psychological  fidelity  cited  most  fi-equently  related  to  the 
finstrations  experienced  during  mission  execution,  and  the  most  fi-equent  source  of  fiirstration 
involved  various  types  of  software  failures. 

CapahUitv  to  perform  ICCC  action  items  usine  FSC.  Details  ofthe  responses  received  in 
this  part  of  the  questionnaire  are  in  Appendix  Table  B-6.  FSC  was  designed  and  developed  to 
permit  the  students  to  perform  22  action  items  or  tasks  derived  fi-om  the  ICCC  lesson  on  light 
Infantry  offensive  operations  in  an  urban  environment.  Overall,  a  majority  ofthe  students 
(69.1%)  indicated  they  could  perform  these  action  items  moderately  well  or  very  well  (the  two 
hipest  rating  scale  categories  in  a  four-point  rating  scale).  The  mean  (and  standard  deviation) 
rating  scale  values  of  the  five  highest  rated  action  items  were:  Develop  a  course  of  action  for  a 
light  company  team,  3.5  (0.6);  Determine  own  force  potential  combat  power,  3.3  (0.7);  Select  a 
course  of  action,  3.3  (0.6);  Analyze  enemy  situation,  3.2  (0.7);  and  Issue  a  fiago  (a  fiagmentary 
order),  3.2  (0.8).  Several  action  items  were  not  performed  using  the  FSC.  The  mean  (and 
s^(^d  deviation)  rating  scale  values  for  the  three  lowest  rated  action  items  were:  Analysis  of 
civilian  considerations,  2.0  (0.9);  Issue  a  company/team  OPORD  (operations  order),  2.2  (0.7); 
and  Issue  an  OPORD  for  Infantry  company,  2.2  (0.8).  When  asked  to  indicate  whether  these  *22 
design  capabilities  of  the  FSC  simulation  should  be  kept  in  future  versions  of  the  simulation,  the 
students  generally  responded  in  the  affirmative.  Averaged  over  all  22  action  items,  98.7  percent 
of  the  participants  indicated  the  current  design  features  of  FSC  should  be  retained  in  future 
versions  of  the  game. 

Additional  ICCC  actions  Items  that  should  be  incorporated  in  future  versions  of  FSC. 
Details  of  the  ratings  and  rankings  provided  in  this  part  of  the  questiormaire  are  in  Appendix 
Table  B-7.  Overall,  66.4  percent  of  the  students  indicated  they  believed  future  versions  ofthe 
FSC  simulation  should  permit  performance  of  17  listed  ICCC  action  items  that  were  not 
currently  progranuned  into  the  game.  However,  the  participants  were  quite  discriminating  in  this 
assessment.  A  very  high  majority  (88.0%  or  more)  indicated  future  versions  should  incorporate 
the  three  following  ICCC  action  items:  Apply  emerging  threat  operations  to  tactical  plannings 
Defend  a  company/team  battle  position,  and  Plan  offensive  fire  support  for  a  mechanized 
Infantry  company/team.  On  the  other  hand,  fewer  than  33  percent  of  the  students  elected  not  to 
endorse  the  incorporation  of  two  action  items  into  future  versions  of  FSC.  These  items  were: 
Conducting  TEWT  (training  exercise  without  troops)  and  Constructing  a  company  METL 
(mission  essential  task  list). 
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Common  responses  to  onen-ended  questions  in  the  FSC  Questionnaire,  A  series  of 
questions  in  the  last  part  of  the  FSC  Questionnaire  asked  students  to  provide  written  comments 
to  address  three  broad  areas  of  concern.  The  comments  received  are  summarized  in  the 
following  numbered  paragraphs. 

(1)  Does  high  quality  training  using  FSC  depend  on  an  experienced  instructor?  Can  you 
learn  as  much  using  FSC  on  your  own?  The  objective  of  these  two  questions  was  to  ascertain 
whether  students  thought  they  could  receive  hi^  quality  training  by  playing  the  FSC  game  on 
their  own,  without  the  assistance  of  the  small  group  instructor  or  other  subject  matter  experts. 
However,  it  was  evident  that  the  students  responded  to  these  questions  in  the  context  of  only  the 
initial  demonstration  and  train-up  session.  Thirteen  of  the  25  students  who  answered  this 
question  indicated  that  training  to  use  FSC  would  be  difficult  without  an  experienced  instructor 
and  twelve  said  they  could  learn  to  play  FSC  on  their  own.  From  these  responses,  it  is  unclear  if 
they  felt  they  could  have  learned  more  about  mission  execution  if  an  instractor  were  present 
during  subsequent  sessions  devoted  to  playing  FSC. 

(2)  Identify  what  you  most  liked  and  what  you  most  disliked  about  the  following  specific 
aspects  of  FSC.  (a)  The  user  manual:  While  ten  participants  indicated  the  user  manual  was 
either  easy  to  use,  clear  and  complete,  or  user  friendly,  three  other  students  indicated  the  user 
manual  needed  either  more  examples,  greater  depth,  or  more  pictures,  and  another  six  reported 
they  never  used  the  manual,  (b)  The  training  provided for  using  FSC:  Three  said  they  had 
enough  time  with  the  game  and  it  was  easy  to  use,  but  twelve  indicated  they  did  not  have  enough 
time  to  play  the  game,  (c)  The  interface  with  FSC  during  mission  planning  and  during  mission 
execution  :  Eight  students  most  liked  the  planning  phase  of  the  game  and  four  most  liked  being 
able  to  implement  fragmentary  orders  during  mission  execution,  but  three  each  most  disliked  the 
fact  that  the  allocation  of  combat  power  could  not  be  changed  without  restarting  the  entire 
planning  phase  and  the  requirements  associated  with  using  planning  codes  (“go  codes”)  to 
determine  the  timing  of  blue  force  movements,  (d)  Finally  and  more  generally,  identify  three 
features  you  liked  best  and  three  features  you  liked  least  about  FSC:  While  the  best  liked 
features  of  FSC  by  at  least  fovir  students  each  were  the  graphics  and  the  realistic  training 
provided  by  FSC,  the  least  liked  feature  of  the  game  by  three  students  each  were  that  blue  force 
Soldiers  did  not  respond  properly  or  realistically,  situation  reports  were  incomplete  or  not 
available  when  needed,  and  software  foiled  during  mission  execution. 

(3)  In  spite  of  the  possible  limitations  you  may  have  encountered  with  FSC,  what  are  its 
advantages  or  potential  for  improving  the  decision-making  skills  of  individuals  in  ICCC? 
Twenty-four  students  provided  written  responses  to  this  question.  Four  mentioned  that  the  FSC 
facilitated  appropriate  decision-making  during  planning  and  mission  execution.  Five  students 
mentioned  foe  importance  of  experiencing  foe  planning  and  execution  phases.  At  least  two  each 
listed  foe  following:  foe  advantages  of  training  basics  being  reinforced  with  FSC,  foe  chance  to 
respond  to  unforeseen  events,  being  able  to  implement  fragmentary  orders,  making  low-cost 
mistakes,  synchronization  of  assets  on  foe  battlefield,  or  using  FSC  to  train  jumor  leaders. 
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Comparison  of  FSC  Ratings  for  Involvement  and  for  Overall  Training  Value  with  Pre-FSC 
and  Post-FSC  Measures 

Mean  ratings  obtained  for  involvement  and  for  overall  training  value  were  examined  for 
possible  relationships  with  the  pre-FSC  and  post  FSC  measiues.  There  were  no  relationships 
found  between  these  two  FSC  Questionnaire  variables  and  pre-FSC  measures  of  general 
cognitive  ability,  pre-FSC  and  post-FSC  measures  of  decision-making  style,  or  post-FSC 
measures  of  Janus  performance.  However,  four  sigmficant  and  meamngfiil  relationships  were 
found  involving  pre-FSC  measures  obtained  with  the  biographical  questionnaire.  As  shown  in 
Table  5,  the  higher  the  participants’  percentage  of  duty  time  in  company-level  assignments,  the 
higher  their  mean  ratings  for  involvement  with  and  overall  training  value  of  FSC.  In  contrast, 
the  higher  the  participants’  percentage  of  duty  time  spent  below  company  echelon,  the  lower 
their  mean  ratings  for  these  two  parts  of  the  FSC  Questionnaire. 


Table  5 

Association  of  FSC  Ratings  with  Biographical  Data 


Percentage  Duty  Time 

Mean  FSC  Involvement 
Rating 

Mean  FSC  Overall 
Training  Value  Rating 

At  Company  Echelon 

+0.413 

+0.460 

Below  Company  Echelon 

-0.417 

-0.537 

Note:  Associations  were  quantified  using  Spearman’s  rho  correlation  coefficients. 


Description  of  Other  Results  Pooled  Over  FSC  and  Control  Groups 

Additional  results  were  found  by  pooling  over  the  data  collected  from  students  in  the  FSC 
and  control  groups.  While  these  results  are  not  relevant  to  the  evaluation  of  FSC  training 
effectiveness,  they  are  of  value  for  developing  a  better  imderstanding  of  the  meaning  and 
interrelationship  of  the  various  measures  obtained  during  the  evaluation.  They  are  reported  here 
because  they  may  be  of  benefit  for  future  research  in  this  and  related  areas,  as  well  as  for  military 
personnel  responsible  for  the  training  of  small  group  leaders. 

Measures  of  Performance  During  the  Janus  Exercise 

Several  potential  relationships  involving  the  performance  of  the  36  students  who 
participated  in  the  Janus  exercise  were  examined  using  Spearman’s  correlation  coefficients. 
Results  showed  two  significant  relationships  among  the  duee  Janus  dependent  variables.  As  the 
mean  percentage  of  adaptable  responses  in  the  Janus  exercise  increased,  there  was  a  decrease  in 
the  percentage  of  trials  on  which  a  prompts  was  provided  (Spearman’s  rho  =  -.351)  and  in  the 
overall  number  of  tactical  errors  (rho  =  -.750). 

Potential  relationships  between  dependent  measures  from  the  Janus  exercise  and  pre-FSC 
and  other  post-FSC  measures  were  also  examined.  Only  three  meaningful  and  significant  results 
emerged  from  these  analyses,  all  involving  items  from  the  biographical  questionnaire.  As  the 
percentage  of  duty  time  at  the  company  level  increased,  the  percentage  of  adaptable  responses 
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for  Janus  tasks  increased,  rho  =  .372,  but  as  the  percentage  of  duty  time  assigned  to  positions 
below  company  level  increased,  the  percentage  of  adaptable  responses  decreased,  rho  —  -.33 1 .  It 
was  also  shown  diat  as  the  percentage  of  duty  time  assigned  to  positions  below  company  level 
increased,  the  number  of  doctrinal  errors  during  the  Janus  exercise  increased,  rho  -  .330. 


Measures  of  General  Cognitive  Ability  and  Decision-Making  Style 

The  measure  of  general  cognitive  ability  was  not  meaningfully  or  significantly  related  to 
any  other  variables  considered  in  this  research.  The  median  score  on  the  Wonderlic  Personnel 
Test  by  participants  in  this  research  (25)  is  higher  than  that  obtained  by  high  school  graduates 
(21),  but  lower  than  that  obtained  by  college  graduates  (29),  where  both  of  the  latter  normative 
scores  were  determined  from  research  using  large  samples  of  participants  from  many 
organizations  across  the  U.S.  From  a  purely  measurement  perspective,  it  must  be  noted  that  die 
correlation  coefficients  found  among  the  measures  of  general  cognitive  ability  and  types  of 
decision-making  style  were  not  found  to  be  significant. 

Paired  sample  t-tests  were  used  to  compare  the  pre-FSC  and  post-FSC  measures  of  the 
three  decision-making  styles.  These  tests  showed  a  difference  for  only  die  regret-based 
emotional  style  of  decision  making,  t(53)  =  2.37.  The  mean  (and  standard  deviation)  scores  were 
45.9  (1 1 . 1)  on  the  pre-FSC  measure  and  43.7  (1 1 .2)  on  the  post-FSC  measure.  Averaged  over 
pre-FSC  and  post-FSC  measures,  the  mean  scores  of  students  for  the  analytic  decision-making 
style  (72.6)  and  the  intuitive  style  (65.3)  were  higher  than  those  reported  by  Nygren  (2000)  for  a 
large  sample  of  college  students  (W.8  and  61.3,  respectively).  On  the  other  hand,  the  pre-FSC 
measure  of  regret-based  emotional  decision-making  style  (45.6)  was  lower  than  that  for 
Nygren’s  sample  (58.5),  and  it  was  even  lower  (43.0)  after  about  9  weeks  of  instruction  in  the 
ICCC. 

Other  meaningful  and  significant  relationships  were  foimd  between  measures  of  decision¬ 
making  style  and  measures  derived  fi-om  die  biographical  questionnaire,  but  only  for  the  regret- 
based  emotional  style  of  decision  making.  For  51  students  providing  useful  data,  regret-based 
emotional  decision  style  was  negatively  correlated  with  the  percentage  of  duty  time  spent  at 
echelons  below  company  level  {rho  —  -.344  for  the  pre-FSC  measure  and  -.373  for  the  post-FSC 
measure).  On  the  other  hand,  regret-based  emotional  decision  style  was  positively  related  to  the 
percentage  of  duty  time  spent  in  a  staff  position  {rho  =  .349  for  the  pre-FSC  measure  and  .379 
for  the  post-FSC  measure).  Finally,  for  the  22  participants  who  reported  spending  non-zero  time 
playing  commercial  computer  video  games,  the  regret-based  emotional  decision  style  was 
negatively  correlated  with  the  reported  number  of  hours  spent  playing  computer  video  games  (- 
.602  for  the  pre-test  measure  and  -.659  for  the  post-test  measure). 

DISCUSSION 

Results  Related  Directly  to  FSC  Traimn£  Effectiveness 

The  officers  who  played  FSC  generally  endorsed  the  use  of  this  computer-based  game  for 
the  course.  Through  their  responses  to  items  in  the  FSC  Questionnaire,  these  officers  indicated 
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(a)  they  became  personally  involved  in  the  game’s  simulated  mission  environment,  (b)  the  three 
types  of  scenarios  they  played  were  challenging,  (c)  the  fidelity  or  realism  of  simulated 
battlefield  events  and  actions  was  quite  good,  and  (d)  playing  the  game  permitted  them  to 
practice  many  of  the  action  items  or  tasks  that  would  enable  them  to  meet  the  objectives  of  the 
course.  Given  these  generally  favorable  opinions  of  the  game’s  attributes,  it  was  not  surprising 
that  they  expressed  highly  positive  opinions  concerning  the  overall  training  value  of  FSC. 

However,  there  also  were  some  caveats  in  the  ratings  provided  by  those  who  played  FSC. 
T^e  majority  indicated  (a)  they  had  to  focus  their  attention  on  their  interactions  with  the  game’s 
display  and  control  devices  rather  than  on  the  experiences  being  created  by  the  game,  (b)  they 
did  not  have  sufficient  opportunities  to  adapt  to  uncertain  conditions  during  simulated  tactical 
missions  and  (c)  playing  FSC  did  not  necessarily  permit  them  to  leam  anything  about  mission 
execution  that  they  had  not  learned  otherwise.  The  last  indicated  opinion  was  especially  evident 
for  the  two  game  scenarios  that  were  played  most  often  by  most  of  the  participants. 

The  implications  of  these  ratings  about  FSC  need  to  be  judged  in  the  context  of  the  current 
state  of  game  development.  The  version  of  FSC  available  for  use  in  this  research  was,  after  all, 
the  initial  release  of  the  game  software  and  should  be  considered  a  prototype.  When  asked  to  do 
so,  most  participants  indicated  that  future  versions  of  the  game  need  to  include  capabilities  for 
actions,  such  as  defensive  operations  and  the  use  of  fire  support,  that  were  not  built  into  the 
current  version  of  FSC. 

Perhaps  most  troubling  for  this  evaluation  of  the  training  effectiveness  of  FSC  was  the 
failure  to  find  any  statistically  reliable  differences  in  Janus  performance  between  officers  who 
played  the  game  and  those  who  did  not.  It  was  anticipated  that  participants  who  had  additional 
training  opportumties  with  FSC  would  demonstrate  superior  adaptive  decision-making  ability 
during  the  Janus  exercise.  This  clearly  was  not  the  case.  However,  several  of  the  more  salient 
reasons  for  there  being  no  difference  between  these  two  groups  are  unrelated  to  the  potential 
trainmg  value  of  playing  FSC.  For  instance,  it  might  be  argued  that  the  Janus  exercise  was  not  a 
valid  measure  of  an  officer’s  ability  to  plan  and  execute  a  simulated  mission.  We  reject  this 
potential  explanation.  The  Janus  exercise  was  created  specifically  for  our  evaluation  research 
and  was  uniquely  designed  to  assess  a  player’s  ability  to  adapt  his  decision  making  to  emerging 
battlefield  conditions. 

We  argue  that  the  most  obvious  reasons  for  finding  no  group  difference  in  Janus 
performance  was  the  limited  opportunity  for  participants  to  play  the  FSC  game  and  the  absence 
of  controlled  procedures  for  using  the  game.  Observations  during  sessions  devoted  to  playing 
FSC  imderscored  both  the  limited  time  available  for  individuals  to  actively  play  FSC  and  the 
varying  procedures  used  for  controlling  which  scenarios  were  played  during  any  given  session. 
Likewise,  ratings  and  written  responses  by  officers  in  the  FSC  group  showed  that  they  believed 
they  did  not  have  enough  time  to  play  FSC.  ^ 


We  wish  to  emphasize  that  the  absence  of  controlled  procedures  for  using  FSC  and  the  limited  amounts  of  time 
available  to  play  FSC  were  due  to  constraints  in  the  ICCC  POl,  and  not  to  any  failure  on  the  part  of  those 
responsible  for  the  ICCC  program  or  the  small  group  instructors  responsible  for  the  seminars  in  which  FSC  was 
introduced. 
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Another  major  shortfall  in  the  procedures  used  for  playing  FSC  was  the  general  absence  of 
any  form  of  structured  evaluation  of  the  game  player’s  decision-making  performance.  Army 
training  doctrine  states  that  all  training  must  be  evaluated  to  measure  performance  levels  against 
established  Army  standards  and  to  establish  whether  the  training  standard  has  been  achieved. 
Indeed,  the  Army’s  capstone  training  publication,  Field  Manual  7.0,  Training  the  Force  (DA, 
2002),  goes  so  far  as  to  state:  “training  without  evaluation  is  a  waste  of  time  and  resources”  (p. 
6-4).  The  after  action  review  is  a  common  method  for  evaluating  training.  It  is  a  structured 
process  incorporating  coaching  and  mentoring  provided  by  an  experienced  instructor.  The  after 
action  review  allows  participating  Soldiers  to  discover  for  themselves  what  happened  during  the 
training,  why  it  happened,  and  how  it  can  be  done  better.  Due  principally  to  constraints  built  into 
the  ICCC  program  of  instruction,  the  extensive  after  action  review  capabilities  built  into  FSC 
were  never  used  by  the  game  players.  However,  it  must  be  noted  that  the  developers  of  FSC  did 
not  intend  the  after  action  review  ftuiction  to  be  used  solely  by  the  player.  The  User  Manual 
indicates  that  it  was  to  be  used  by  both  the  player  and  the  instructor  to  examine  the  execution  of 
the  mission  in  detail,  and  to  analyze  and  evalxiate  the  decisions  made  by  the  player. 

Results  Related  to  Other  FSC  Training  Issues 

The  results  of  our  research  highlight  a  variety  of  general  training  issues,  quite  apart  fi-om 
the  issue  of  the  game's  training  effectiveness.  For  example,  FSC  players  indicated  the  three 
tactical  scenarios  used  during  this  research  provided  opportumties  for  different  levels  and 
different  types  of  involvement  and  learning  experiences.  There  was  little  evidence,  however, 
that  these  scenarios  were  designed  to  vary  systematically  the  difficulty  of  the  required  decision¬ 
making  process.  In  future  versions  of  FSC-type  games,  care  should  be  taken  to  identify  what 
specific  types  of  decisions  and  types  of  decision-making  requirements  are  imposed  by  the 
various  scenarios  used  to  drive  the  command  and  control  ftuiction.  If  this  were  done,  the  training 
needs  of  students  could  be  matched  by  the  training  opportunities  provided  by  particular 
scenarios. 

Another  factor  highlights  the  issue  of  the  fidelity  or  realism  of  battlefield  events  and  actions 
portrayed  in  FSC.  The  game  players’  responses  to  queries  about  different  types  of  fidelity 
showed  that  the  artificial  intelligence  software  that  determines  actions  of  the  blue  force  is  more 
important  than  the  quality  of  visual  graphics  used  to  portray  physical  features  of  the  environment 
or  characteristics  of  the  opposing  threat  force.  For  purposes  of  training  decision-making  skills, 
doctrinally  accurate  portrayal  of  the  blue  force  responses  to  orders  of  the  commander  and  of 
tactically  significant  environmental  factors  must  t^e  precedence  over  concerns  for  other  purely 
physical  features  of  the  displayed  battlefield. 

Another  training  issue  addressed  by  the  results  of  our  evaluation  is  the  capability  of  FSC  to 
train  for  the  deliberative  planning  that  occurs  before  a  mission  is  initiated,  as  well  as  for  the  hasty 
planning  that  occuTS  while  a  mission  is  being  executed.  As  currently  developed,  FSC  requires 
that  the  participant  both  create  his  operations  plan  and  monitor  the  execution  of  the  plan. 
Monitoring  the  execution  of  the  operations  plan  is  necessary  to  acquire  the  insights  needed  to 
develop  and  issue  fragmentary  orders  rapidly  to  change  the  original  plan.  Observations  made 
while  FSC  was  being  played  showed  that  during  many  time-limited  sessions,  some  game  players 
spent  so  much  time  planning  a  mission  that  there  was  little  if  any  time  left  to  execute  the 
mission.  Much  of  the  time  required  to  develop  an  operations  plan  was  due  to  a  lack  of 
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familiarity  with  how  to  do  so  in  the  FSC  game-playing  environment.  However,  the  acts  of 
analyzing  a  mission  and  developing  an  operations  plan  are  inherently  time-consuming  cognitive 
processes  that  can  be  learned  using  other  peda.gogical  methods  that  already  exist. 

We  believe  the  FSC’s  unique  potential  contribution  to  training  future  company 
commanders  is  not  to  augment  die  knowledge-based  instructions  already  adequately  provided. 

In  fact,  the  company  command  segment  of  the  ICCC  is  already  structured  to  provide  classroom 
instruction  and  practical  exercises  designed  to  teach  students  how  to  develop  tactically  and 
doctrinally  sound  operational  plans  and  to  issue  operations  orders  for  an  Infantry  company. 
Rather,  we  believe  the  player  of  FSC  should  be  given  the  opportunity  to  learn  ej^erientially 
about  the  possible  consequences  of  executing  an  operations  order  and  making  hasty  changes  to 
that  order.  Experiential  learning  builds  on  previous  knowledge  and  is  a  prerequisite  for  the 
development  of  tactical  expertise.  Future  versions  of  FSC  could  consider  an  optional  mode  of 
play  that  does  not  require  the  act  of  deliberately  planning  a  mission.  Rather,  the  player  might 
simply  be  given  an  operations  order  prepared  in  advance  to  highlight  specific  training  objectives 
(i.e.,  decision  points  and  their  potential  consequences).  The  objective  for  the  player  would  then 
be  to  monitor  the  execution  of  that  operations  order  to  acquire  skills  necessary  for  making  the 
hasty  decisions  required  to  respond  to  unanticipated  conditions  of  the  battlefield  environment. 
Using  such  an  option,  the  number  of  experiential  learning  opportunities  for  the  FSC  player 
would  increase  substantially  over  what  is  now  possible. 

A  final  training  issue  raised  by  this  research  is  related  to  whether  FSC  is  equally  well  suited 
for  all  students  in  an  ICCC  class.  The  results  are  quite  clear  in  showing  that  the  participants’ 
ratings  of  involvement  in  and  overall  training  value  of  FSC  was  positively  related  to  the 
percentage  of  their  reported  duty  time  in  company-level  assignments  and  negatively  related  to 
the  percentage  of  their  duty  time  in  assignments  below  company  echelon.  These  results  suggest 
that  the  training  value  of  FSC  may  vary  with  the  military  experience  players  bring  to  ICCC.  If 
trae,  these  results  suggest  that,  at  a  minimum,  the  relative  difficulty  of  mission  scenarios  should 
vary  to  accommodate  the  varying  backgrounds  of  the  students.  For  example,  scenarios  could 
vary  the  number  of  assets  that  the  student  has  while  he  plans  and  executes  a  mission  (e.g.,  fewer 
assets  than  the  three  Infantry  platoons  and  mortar  section  typically  available  to  an  Infantry 
company  commander).  Indeed,  it  could  be  useful  to  create  a  platoon  leader  version  of  FSC.  At 
the  other  extreme,  scenarios  could  increase  the  number  of  assets  whose  actions  need  to  be 
monitored  and  synchronized  and  otherwise  increase  the  complexity  of  the  battlefield 
environment.  In  any  case,  the  goal  would  be  to  provide  a  student  initially  with  a  level  of 
complexity  that  matches  the  breadth  and  depth  of  his  military  experience.  Then,  in  line  with  the 
crawl-walk-run  approach  of  training  challenges,  the  difficulty,  uncertainty,  and  realism  of  the 
scenario  could  be  increased  as  the  student  gains  mastery  of  successively  more  complex  missions 
that  demand  an  expert  company  commander’s  vision  of  the  battlespace. 

Results  Related  to  Measurement  Issues 

Finally,  several  of  the  methods  employed  during  this  research  yielded  results  that  suggest 
they  need  to  be  further  investigated  for  their  potential  value  dunng  fixture  evaluations  of 
computer  games.  For  example,  the  use  of  the  Janus  Scenario  Assessment  Checklist  in  this 
research  permitted  a  direct  assessment  of  a  player’s  ability  to  adapt  to  changing  and  imcertain 
battlefield  conditions.  The  same  types  of  decision-making  assessment  techniques  need  to  be 
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developed  for  FSC.  The  direct  measure  of  decision-making  adaptability  can  serve  to  calibrate 
the  quality  and  amount  of  training  acquired  by  students,  as  well  as  measme  the  capability  of 
these  players  to  transfer  the  skills  they  have  learned  to  other  simulated  or  live  tactical 
environments.  While  Janus  is  a  resource  intensive  method  for  assessing  training  effectiveness, 
its  use  is  required  only  during  relatively  short-duration  evaluations  of  much  longer-term  efforts 
to  design  and  develop  training  methods.  Furthermore,  the  development  and  use  of  additional 
objective  criterion  measures,  such  as  those  that  track  the  timing  and  sequencing  of  adaptive 
decisions,  would  still  further  enhance  the  value  that  can  be  derived  from  these  types  of  methods 
for  evaluating  training  effectiveness. 

While  a  measure  of  general  cognitive  ability  was  not  shown  to  be  useful  for  predicting  the 
effects  of  playing  FSC,  there  is  some  suggestion  that  measures  of  decision-making  style  could 
prove  to  be  of  value.  The  fact  that  measures  obtained  for  the  three  types  of  decision-making 
style  were  mutually  independent  suggests  that  individuals  may  score  differently  on  one  scale 
than  the  others.  There  is  considerable  discussion  in  Army  doctrine  for  the  distinction  between 
analytical  and  intuitive  styles  of  decision  making.  Field  Manual  6.0,  Mission  Command: 
Command  and  Control  of  Army  Forces  (DA,  2003),  specifies  that  both  styles  of  decision  making 
are  useful.  That  publication  also  notes  it  may  be  appropriate  to  shift  from  more  analytical  styles 
in  deliberative  decision-making  situations  to  more  intuitive  styles  during  situations  that  require 
hasty  decision  making.  Participants  in  our  research  scored  initially  quite  high  on  measures  of 
their  propensity  to  use  analytical  and  intuitive  decision-making  styles.  Consequently,  there  may 
have  been  a  “ceiling  effect”  with  the  Decision  Making  Style  Inventory  that  lessened  the  chance 
for  post-FSC  meastnes  to  show  an  increase  for  either  of  these  decision-making  styles. 
Furthermore,  there  were  too  few  participants  to  permit  contrast  of  those  who  score  high  or  low 
on  each  of  the  three  scales  for  decision-making  style.  Other  methods  (e.g.,  the  Adaptive- 
hmovation  scale  developed  by  Kirton,  1994)  need  to  be  examined  for  their  relevance  in 
examining  different  styles  of  decision-making  and  to  evaluate  training  methods  to  optimize  the 
use  of  different  styles. 

The  fact  that  measures  for  the  regret-based  emotional  decision-making  style  tended  to 
decrease  during  the  company  command  segment  of  the  ICCC  suggests  that  the  course  achieves 
one  of  its  objectives.  After  just  ten  weeks  in  the  course,  students  seem  to  acquire  a  greater 
propensity  for  making  decisive  decisions  without  experiencing  excessive  doubt  about  their 
actions  or  dwelling  excessively  upon  the  risk  associated  with  them.  The  relationships  found 
between  measures  of  regret-based  emotional  style  of  decision  making  and  the  number  of  hours 
spent  playing  commercial  video  games  is  quite  interesting  and  may,  in  part,  serve  to  validate 
measures  obtained  using  the  scale.  The  relationship  suggests  that  regret-based  emotional 
individuals,  who  tend  to  score  high  in  both  personal  and  judgmental  self-doubt  and  low  in  risk 
seeking  and  self  esteem,  spend  relatively  little  time  playing  video  games,  and  vice  versa.  Those 
who  never  play  video  games  have  a  much  higher  range  of  regret-based  emotional  style  scores. 
Indeed,  a  point  worth  emphasizing  is  the  fact  that  the  majority  of  the  participants  in  this  research 
reported  that  they  never  play  video  games.  The  more  commonly  held  opinion,  that  most  Soldiers 
spend  many  hours  playing  video  games,  is  given  as  one  justification  for  promoting  the  adoption 
of  video  games  as  a  method  for  traming  cognitive  abilities  required  of  Soldiers. 
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A  finEl  point  needs  to  be  made  for  investigEting  further  the  vElue  of  determining  an 
officer’s  propensity  for  engEging  in  a  regret-bEsed  emotionEl  style  of  decision  msking.  Officers 
in  our  resesreh  who  hEd  spent  relEtively  Isrge  amounts  of  time  in  assignments  below  company 
level  scored  low  on  the  regret-based  emotional  scale,  suggesting  they  would  tend  to  score  high  in 
risk  taking  and  self  esteem  and  low  in  both  personal  and  judgmental  self-doubt.  In  contrast, 
officers  who  have  spent  relatively  large  amounts  of  time  in  battalion  or  brigade  staff  positions 
scored  high  on  the  regret-based  emotional  scale,  indicating  they  would  tend  to  score  low  in  risk 
taking  and  self  esteem  and  high  in  both  personal  and  judgmental  self-doubt.  Clearly,  these 
restate  suggest  the  assignment  history  of  officers  may  influence  their  predominant  styles  for 
decision  making,  and  that  in  turn  could  affect  the  methods  and  procedures  most  appropriate  for 
training  adaptive  decision-making  behaviors. 


CONCLUSION 

The  absence  of  statistically  reliable  differences  in  criterion  measures  of  adaptive  decision- 
niaking  behaviors  between  participants  who  played  FSC  and  a  control  group  of  participants  who 
did  not  play  FSC  would  be  accepted  generally  as  evidence  that  FSC  did  not  contribute  to 
officers’  acquisition  of  requisite  cognitive  skills.  However,  the  failure  to  find  differences 
between  groups  for  this  criterion-based  measure  of  training  success  is  more  likely  due  to 
constraints  in  the  time  available  and  the  procedures  used  for  playing  the  game  rather  than  to 
some  fault  in  the  game  itself.  Officers  participating  in  our  research  clearly  did  not  have 

time  for  playing  FSC.  In  addition,  because  these  officers  did  not  participate  in  any 
form  of  perfoimance  evaluation  after  they  played  FSC,  it  is  impossible  to  determine  objectively 
what,  if  anything,  was  learned  while  playing  foe  game. 

While  shortcomings  in  experimental  procedures  may  have  been  confounded  comparisons 
between  participants  who  played  FSC  and  those  who  did  not,  officers  in  foe  FSC  group  had 
enough  exposure  to  foe  game  to  form  definitive  opinions  about  it  and  its  value  for  training.  The 
results  suggest  that  these  officers  believed  FSC  provided  them  with  experiences  that  were  of 
value  both  to  themselves  and  to  other  students  in  foe  ICCC.  Opinions  expressed  by  officers  who 
played  FSC  also  identified  specific  changes  that  should  be  made  in  future  versions  of  foe  game. 
In  addition,  other  measures  showed  foe  amount  and  types  of  military  experience  acquired  prior  to 
enrollment  in  foe  ICCC  were  related  to  measures  of  perceived  involvement,  training  value, 
adaptive  decision-making  behavior,  and  decision  making  style.  These  latter  findings  suggest  that 
future  versions  of  FSC-type  computer  games  should  consider  differences  in  experiences  foe 
player  has  previously  acquired  during  his  military  career. 
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A-1 


Biographical  Information  Questionnaire 


Name _ _ _ 

Please  fill  in  the  blanks  or  check  the  aooropriato  ra-cponses. 

1.  What  is  your  age?  Years _ 

2.  What  is  your  rank? _ 


3.  Source  of  Commissioning?  (Check  one)  ROTC 
Direct  Commission _  Other 


OCS 


USMA 


4.  Time  in  service:  Years_ 

Time  in  rank:  Years 


Months, 

Months 


5.  If  you  are  Army,  what  is  your  branch,  and  if  not  Army,  what  is  your  service? _ 

6.  If  Infantry,  what  percentage  (0  to  100%)  of  your  Infantry  experience  is  Mechanized? 

7.  If  Infantry,  have  you  been  awarded  the  Expert  Infantryman  Badge?  Yes _  No 


8.  Beginning  with  your  assignment  before  coming  to  ICCC,  identify  the  duty  positions  you  have 
t  eld,  the  units  in  which  yoi  held  these  positions,  and  the  duration  of  the  assignments. _ 

Duty  Position  Unit  Designation  Location  of  Unit  #  Months  in 

- — - - - - -  Duty  Position 


9.  What  Army  training  courses  have  you  completed?  Check  aH  that  apply. 

Airborne _  BFV  Leader  Course _  Ranger _  Air  Assault 

IMPOC _  Pathfinder  School _ 

Other(s) _ 

10.  Were  you  prior  enlisted?  Check  “Yes"  or  “No."  Yes_  No 

If  "Yes,"  which  NCO  course(s)  did  you  complete?  Check  all  that  apply 
PLDC _  BNCOC _  ANCOC _ 

Other(s) _ _ _ 

1 1 .  Have  you  been  on  a  rotation  to  any  of  the  following  combat  training  centers? 


Combat 

Training 

Center 


NTC 


Check  If  “Yes,”  what  was  your 
“Yes”  or  “No”  duty  position? 


If  “Yes,”  did  you  take  part 
in  live  fire  exercises? 
Check  “Did”  or  “Did  Not 


Did 

Did  Not 

Did 

Did  Not 

Did 

Did  Not 

12.  Have  you  had  experiences  in  military  operations  Outside  the  Continental  United  States? 
Check  “Yes”  or  “No.”  Yes _  No _ 


If  “Yes,”  indicate  the  location  of  these  experiences  by  checking  all  the  options  that  apply  and 
then  provide  the  other  information  requested.  If  “No.”  go  directly  to  the  next  item  (#13). 

I  *  -  I  I  ^  _  I  A _ ■ _ A.  \ 


Location 

Duty  Position 

Duration 

(Months) 

Combat 

Experience 

stability  Ops 
Experience 

Afghanistan 

mmSm 

■SB 

Yes 

Bosnia 

■SB 

No 

Yes 

Kosovo 

No 

Desert  Storm 

HB 

BiSH 

Other(s) 

BBS 

fl!SB 

bESWHI 

HBSI 

BSSSB 

3.  Have  vou  had  prior  experience  with  the  following  events?  C 

heck  all  that  apply. 

Event 

Platoon 

Level 

Company 

Level 

Bn  Staff 
Level 

ARTEP  or  external  evaluator. 

Command  of  a  unit  undergoing  an  ARTEP  or  external  evaluation 

Coordination  with  engineers 

Coordination  with  medics. 

Coordination  with  armor  unit.  __  . 

Combined  arms  live-fire  exercise. 

In  charge  of  mortar  live-fire.  . 

Ground-to-air  coordination  (fixed  wing). 

Ground-to-air  coordination  (rotary  wing). 

Calling  artillery  rounds. 

Coordination  with  a  fonward  observer 

Responsible  for  casualty  evacuation  (plan  or  execution). 

Issuing  orders. 

_ i 

Logistical  support. 

14.  Have  you  had  experience  with  military  training  simulations  (Janus,  CCTT)?  Yes _  No _ 

If  “Yes,”  please  describe  briefly  or  give  the  names  of  the  simulations. 


15.  How  many  hours  per  week  do  you  play  commercial  computer  games? _ hours  per  week 

List  the  games  you  play  (e.g..  Civilization,  Delta  Force,  Rainbow  6) 


16.  Is  there  any  other  military  experience  we  have  not  covered?  [Use  reverse  side  of  this  page 
if  more  room  is  needed  to  write.] 


[Thank  you  for  completing  this  questionnaire] 


Name 


Decision-Making  Style  Inventory 


We  are  interested  In  how  you  typically  go  about  making  decisions.  Think  about  different  situations  and 
contexts  in  which  you  have  made  decisions  recently.  Then,  using  the  6-point  scale  shown  below,  indicate 
the  degree  to  which  you  agree  or  disagree  with  each  numbered  statement.  Keep  in  mind  that  there  are 
no  light  or  wrong  answers  to  any  of  these  items,  because  there  is  no  single  “besr  way  to  make  every 
decision.  It  is  important  that  you  try  to  answer  alt  questions.  However,  if  you  feel  uncomfortable  with  any 
item,  you  may  choose  to  omit  it. 


1 

2 

3 

4 

5 

6 

strongly 

Moderately 

Slightly 

Slightly 

Moderately 

Strongly 

Disagree 

Disagree 

Disagree 

Agree 

Agree 

Agree 

1. 

2_ 

3_ 

4_ 

5_ 

6_ 

7_ 

8_ 

9_ 

10 

11. 

12 

13 

14 


15 

16. 

17 

18 
19 


20 

21. 

22 


I  feel  that  If  I  plan  my  decisions  carefully  I  will  make  good  decisions. 

In  spontaneous  decision  situations  I  usually  find  that  I  have  good  intuitions. 

I  think  that  I  could  keep  myself  from  worrying  later  if  I  had  made  a  bad  decision. 

In  making  decisions  I  first  try  to  make  a  mental  list  of  all  the  factors  or  attributes  that  will  be 
important  to  my  decision. 

I  can  get  a  good  “feeling"  for  most  decision  situations  very  quickly. 

I  sometimes  spend  too  much  time  hesitating  before  making  decisions. 

Before  I  make  a  decision,  I  like  to  figure  out  the  most  efficient  way  of  studying  it. 

I  feel  that  I  have  a  knack  for  making  good,  quick  decisions. 

Before  I  make  a  decision,  I  think  about  whether  others  will  approve  or  disapprove  of  it. 

I’m  very  rational  when  it  comes  to  evaluating  risky  options. 

I  think  that  relying  on  one’s  “gut  feelings"  is  a  sound  decision  making  principle. 

I  tend  to  be  someone  who  worries  a  lot  over  decisions  I’ve  made. 

In  making  decisions  I  first  make  a  careful  initial  estimate  of  the  situation. 

There  are  many  common  sense  “rules-of-thumb"  that  I  know  of  that  usually  lead  to  good 
decisions. 

After  making  a  decision,  I  find  that  I  often  go  back  and  re-evaluate  the  situation. 

I  try  to  pay  attention  to  past  information  in  making  new  decisions. 

Sometimes  decisions,  even  Important  ones,  are  not  difficult  to  make  because  they  just  “feel"  right. 

I  have  trouble  putting  the  results  of  disappointing  decisions  I’ve  made  behind  me. 

A  good  rule  of  thumb  is  that  the  more  information  I  have  In  making  a  decision,  the  better  that 
decision  will  be. 

Simple  decision  rules  usually  work  best  for  me. 

I  rarely  rehash  old  decisions  I’ve  made. 

In  making  decisions  I  try  to  evaluate  the  importance  of  each  piece  of  infonmation  in  the  decision 
process. 


[Please  continue  on  next  page] 
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Continue  to  use  the  following  rating  scale  for  each  statement 


1 

2 

3 

4 

5 

6 

strongly 

Moderately 

Slightly 

Slightly 

Moderately 

Strongly 

Disagree 

Disagree 

Disagree 

Agree 

Agree 

Agree 

23  _ When  forced  to  make  a  quick  decision,  I  find  that  information  that  readily  comes  to  mind  is  usually 

the  most  useful  in  making  a  choice. 

24  _ Worrying  about  future  decisions  that  I  have  to  make  is  something  I  often  do. 

25  _ I  always  try  to  be  fully  prepared  before  I  begin  working  on  making  a  decision. 

26  _ My  first  reaction  to  a  decision  situation  usually  turns  out  to  be  the  best  one. 

27  _ Many  times  when  I  look  back  on  a  choice  I’ve  made,  I  wish  that  I  would  have  put  more  effort  into 

evaluating  the  alternatives. 

28  In  making  decisions  I  try  to  examine  the  importance  of  the  good  and  bad  points  of  each 
alternative. 

29  _ If  I  can't  decide  what  to  do,  I  go  with  my  "best  guess". 

30  _ When  I  find  out  that  I’ve  made  a  bad  decision  I  feel  a  lot  of  regret. 

31  _ I  like  to  take  a  rational,  systematic  approach  to  making  decisions. 

32  _ When  making  decisions,  my  first  instinct  usually  turns  out  to  be  best. 

33  _ If  I  were  gambling  at  a  casino  I  would  prefer  to  play  simpler  games  like  slot  machines  where  you 

don’t  have  to  concentrate  on  playing  complex  strategies. 

34  _ My  best  decisions  are  those  for  which  I’ve  carefully  weighed  all  of  the  relevant  information. 

35  _ I  let  my  intuition  play  a  big  part  in  most  decisions  I  make. 

36  _ I  generally  don’t  make  very  good  decisions  under  time  pressure. 

37  _ I  generally  rely  on  careful  reasoning  in  making  up  my  mind. 

38  _ I  often  rely  on  my  first  impression  when  making  a  decision. 

39  _ I  sometimes  get  “butterflies”  in  my  stomach  when  I  have  to  make  decisions. 

40  _ I  like  to  make  decisions  in  an  orderly  manner. 

41  _ I  rely  on  my  intuition  in  making  many  of  my  personal  decisions. 

42  _ After  making  a  decision  I  sometimes  worry  about  the  regret  I’ll  feel  if  it  the  outcome  turns  out  to 

be  a  bad  one. 

43  _ Most  important  decisions  in  life  are  complex  and  need  to  be  evaluated  in  a  systematic  way. 

44  _ I  find  that  my  best  decisions  usually  result  from  using  the  “quick  and  easy"  approach  rather  than 

the  “slow  but  sure”  method. 

45  _ Unexpected  bad  outcomes  have  a  greater  impact  on  me  than  do  unexpected  good  outcomes. 


[Thank  you  for  completing  this  inventory.] 
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Name_ _  Date _ 

Based  on  your  experience  executing  missions  with  Full  Spectrum  Command  (FSC),  respond  to  the 
questions  in  Parts  1  and  2  of  this  questionnaire  by  marking  an  "X"  in  the  appropriate  box  of  the  7-point 
scale.  Please  consider  the  entire  7-point  scale  before  making  your  responses. 

P^rt  1.  Involvement  In  the  FSC  Simulated  Mission  Environment 


1 .  How  much  were  you  able  to  control  events  during  the  olannino  phase  of  the  mission? 


NOT  AT  ALL 


_l _ l_ 

SOMEWHAT 


COMPLETELY 


2.  How  much  were  you  able  to  control  events  during  the  execution  phase  of  the  mission? 


NOT  AT  ALL 


SOMEWHAT 


COMPLETELY 


3.  How  responsive  was  the  FSC  environment  to  actions  that  you  initiated? 


NOT 

RESPONSIVE 


MODERATELY 

RESPONSIVE 


COMPLETELY 

RESPONSIVE 


4.  How  natural  did  your  interactions  with  the  FSC  environment  seem  to  be? 


EXTREMELY 

ARTIFICIAL 


BORDERLINE 


COMPLETELY 

NATURAL 


5.  How  involved  were  you  with  events  in  the  FSC  environment? 


NOT  AT  ALL 


SOMEWHAT 


COMPLETELY 


6.  How  natural  were  the  processes  that  controlled  movement  through  the  FSC  environment? 


I  I 


EXTREMELY 

ARTIFICIAL 


[Continue  on  next  page] 


9.  How  much  did  your  experiences  in  the  FSC  environment  seem  consistent  with  your  real  world 
experiences? 


I  I  I _ I _ I _ I _ I - 1 

not  moderately  very 

CONSISTENT  CONSISTENT  CONSISTENT 

10.  How  compelling  was  your  sense  of  people  and  objects  moving  through  space? 

I  I  I  I _ I _ I - 1 - 1 

NOT  AT  ALL  MODERATELY  VERY 

COMPELLING  COMPELLING 

1 1 .  How  compelling  was  your  sense  of  moving  around  inside  the  FSC  environment? 

I  I  I _ I _ I _ I _ I - 1 

NOT  AT  ALL  MODERATELY  VERY 

COMPELLING  COMPELLING 

12.  How  well  could  you  observe  and  examine  people  and  objects  from  multiple  positions  in  the  simulated 
battlefield  environment? 

I  I  I _ I _ I _ I - 1 - 1 

NOT  AT  ALL  SOMEWHAT  EXTENSIVELY 

13.  How  well  could  you  move  or  manipulate  objects  in  the  FSC  environment? 

I _ I  I  I _ I _ I - 1 - 1 

NOT  AT  ALL  SOMEWHAT  EXTENSIVELY 

14.  How  involved  were  you  in  the  experiences  provided  by  the  FSC  environment? 

I  I  I  .  I _ I _ I _ I - 1 

not  mildly  completely 

INVOLVED  INVOLVED  ENGROSSED 

15.  How  quickly  did  you  adjust  to  experiences  provided  by  FSC? 

I _ I  I  I _ I _ I _ I - 1 

NOT  AT  ALL  SLOWLY  WITHIN 

MINUTES 

16.  How  proficient  did  you  feel  moving  in  and  interacting  with  the  simulated  environment  at  the 
conclusion  of  your  experiences  with  FSC? 

I _ I  I  I  I _ I _ I - 1 

not  reasonably  very 

PROFICIENT  PROFICIENT  PROFICIENT 

17.  How  much  did  the  quality  of  the  visual  display  interfere  or  distract  you  from  performing  necessary 
tasks  or  required  activities? 

I _ I  I  I  I _ I _ I - 1 

NOT  AT  ALL  INTERFERED  INTERFERED 

SOMEWHAT  GREATLY 


[Continue  on  next  page] 
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18.  How  much  did  the  input  or  control  devices  interfere  or  distract  you  from  performing  necessary  tasks 
or  required  activities? 


I - 1 - 1 _ I _ I _ I  I  I 

NOT  AT  ALL  INTERFERED  INTERFERED 

SOMEWHAT  GREATLY 

19.  How  well  could  you  concentrate  on  performing  necessary  tasks  or  required  activities  rather  than  on 
the  mechanisms  used  to  perform  those  tasks  or  activities? 

I - \ _ I _ I  I _ I _ 1 _ I 

NOT  AT  ALL  SOMEWHAT  COMPLETELY 

20.  To  what  extent  did  events  occurring  in  or  outside  the  classroom  distract  from  your  experiences  in  the 
FSC  environment? 


1 - 1 _ I _ I _ I _ I _ I _ I 

NOT  AT  ALL  MODERATELY  VERY  MUCH 

21.  Overall,  how  much  did  you  focus  on  using  the  display  and  the  control  devices  rather  than  on  the 
experiences  created  by  FSC  for  commanding  a  company  mission? 

I - 1 _ I _ I _ I _ I  I  I 

NOT  AT  ALL  SOMEWHAT  VERY  MUCH 

22.  Were  you  so  involved  In  commanding  the  simulated  mission  that  you  lost  track  of  fime? 

I - 1 - 1 _ I _ I _ I  I  I 

NOT  AT  ALL  SOMEWHAT  COMPLETELY 

23.  Were  there  moments  during  your  experience  vrith  FSC  when  you  felt  completely  focused  on  the  task 
of  commanding  a  company  mission? 

I - 1 - 1 _ I _ I _ I _ I _ I 

none  occasionally  frequently 

24.  How  did  the  total  duration  of  a  mission  execution  affect  your  sense  of  involvement  in  the  FSC 
experience? 


I - 1 _ I _ I _ I _ I _ I _ I 

DECREASED  NO  EFFECT  ON  INCREASED 

INVOLVEMENT  INVOLVEMENT  INVOLVEMENT 


[Continue  on  next  page] 
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Part  2.  Overall  Training  Value  of  FSC 


1.  Was  using  FSC  a  valuable  learning  experience? 

I  I  I _ I _ I _ I _ I - 1 

NO  SOME  GREAT 

VALUE  VALUE  VALUE 

2.  Did  FSC  provide  the  opportunity  to  practice  your  decision-making  skills? 


NO  SOME  GREAT 

OPPORTUNITY  OPPORTUNITY  OPPORTUNITY 

3.  Did  FSC  allow  you  to  focus  on  the  critical  factors  that  influenced  decision-making  during  simulated 
tactical  operations? 

I _ I _ I  I _ I _ I _ I _ I 

NO  SOME  EXCELLENT 

FOCUS  FOCUS  FOCUS 

4.  Do  you  think  your  training  with  FSC  will  improve  your  ability  to  make  rapid  decisions  during  future 
tactical  operations? 


I  I  I _ I _ I _ I _ ^ _ I _ I 

WILL  NOT  MAY  WILL  DEFINITELY 

IMPROVE  IMPROVE  IMPROVE 

5.  Do  you  think  your  training  with  FSC  will  make  you  more  confident  in  your  ability  to  make  appropriate 
decisions  during  future  tactical  operations? 

I _ I _ I _ I _ I _ I - 1 - 1 

NOT  more  SOMEWHAT  MORE  MUCH  MORE 

CONFIDENT  CONFIDENT  CONFIDENT 

6.  How  many  opportunities  did  FSC  give  you  to  adapt  to  uncertain  conditions  during  simulated 
tactical  operations? 


NO  SOME  MANY 

OPPORTUNITIES  OPPORTUNITIES  OPPORTUNITIES 

7.  Do  you  think  your  training  with  FSC  will  make  you  more  confident  in  your  ability  to  adapt  to  uncertain 
conditions  during  ftrture  tactical  operations? 

I _ I _ I _ I _ I _ I - 1 - 1 

NOT  MORE  SOMEWHAT  MORE  MUCH  MORE 

CONFIDENT  CONFIDENT  CONFIDENT 

8.  How  challenging  were  the  overall  experiences  provided  by  FSC? 


not  moderately  very 

CHALLENGING  CHALLENGING  CHALLENGING 

[Continue  on  next  page] 
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9.  How  would  you  describe  the  amount  of  time  you  spent  using  FSC? 

I _ I _ I _ I _ I _ I _ I _ I 

TOO  LITTLE  ABOUT  RIGHT  TOO  MUCH 

10.  How  desirable  is  it  to  use  a  simulation  capability  like  FSC  to  experience  the  possible  consequences 
of  executing  a  tactical  plan? 

I _ I _ I _ I _ I _ I _ I _ I 

NOT  AT  ALL  SOMEWHAT  VERY 

DESIRABLE  DESIRABLE  DESIRABLE 


{Continue  on  next  page] 
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Part  3.  Training  Value  of  Specific  FSC  Scenarios _ _ 

Answer  the  questions  in  the  table  below  by  circling  the  appropriate  alternative. 


Question 

FSC  Scenario 

Recon 

Breach 

Cordon 

and 

Search 

How  many  times  did  you  piay  this 
scenario? 

0 

1 

2 

3  or  more 

0 

1 

2 

3  or  more 

0 

1 

2 

3  or  more 

If  you  played  the  scenario,  was  it 
challenging? 

Yes... No 

Yes... No 

Yes. ..No 

If  you  played  the  scenario,  how 
many  times  did  you  modify  your 
plan  the  last  time  you  played  it? 

0 

1 

2 

3 

4 

5  or  more 

0 

1 

2 

3 

4 

5  or  more 

0 

1 

2 

3 

4 

5  or  more 

Did  you  learn  anything  about 
mission  execution  with  FSC  that 
you  had  not  learned  otherwise? 

Yes... No 

Yes... No 

Yes.. .No 

If  your  answer  in  the  last  row  of  the  table  was  “Yes”  for  any  scenario,  indicate  what  was  learned  by  using 
FSC. 


Was  the  learning  unique  to  FSC  more  significant  for  one  scenario  than  for  others?  Circle:  Yes  or  No  . 
If  Yes,  indicate  how  the  scenarios  differed  in  this  regard. 


[Continue  on  next  page] 
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Part  4.  FSC  Fidelity 


The  FSC  simulation  can  be  assessed  in  terms  of  different  types  of  fidelity: 

•  Physical  fidelity  -  The  degree  to  which  the  displayed  physical  environment  is  realistic.  Do 
individuals  look  and  move  like  real  people?  Are  buildings  realistic?  Does  the  terrain  look  realistic? 

•  Tactical  fidelitv  —  The  degree  to  which  Blue  and  Enemy  force  elements  behave  In  accordance  with 
doctrine  or  reasonable  expectations. 

Blue  force  -  Does  the  Blue  force  react  according  to  doctrine?  Does  it  react  in  a  timely  manner? 
Enemy  force  —  Does  the  Enemy  force  react  as  you  would  expect  an  enemy  to  react  given  the 
activities  of  the  Blue  force? 

•  Psychological  fidelitv  -  The  degree  to  which  you  become  involved  in  your  role  as  Company 
Commander  during  mission  execution. 

In  the  table  below,  provide  your  assessment  of  FSC  in  terms  of  these  types  of  fidelity. 


Identify  FSC  features 
that  support  your  assessment 


Type  of  Fidelity 

Is  this 
fidelity 
important 
in  FSC? 

How  good 
was  this 
fidelity  in 
FSC? 

Physical 

Yes 

Somewhat 

No 

Excellent 

Good 

Adequate 

Poor 

Inadequate 

Tactical 
-  Blue  Force 

Yes 

Somewhat 

No 

Excellent 

Good 

Adequate 

Poor 

Inadequate 

Tactical 
-  Enemy  Force 

Yes 

Somewhat 

No 

^cellent 

Good 

Adequate 

Poor 

Inadequate 

Psychological 

Yes 

Somewhat 

No 

Excellent 

Good 

Adequate 

Poor 

Inadequate 

Rank  order  the  four  types  of  fidelity  in  terms  of  their  relative  importance.  Use  “1"  to  indicate  most 
important  and  “4"  to  indicate  least  important 


Physical _ 

[Continue  on  next  page] 


Tactical  -  Blue 


Tactical  -  Enemy. 


Psychological . 
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Part  5.  Capability  to  Perform  iCCC  Action  Items  Using  FSC 

FSC  was  designed  so  you  could  perform  the  ICCC  action  items  listed  below.  For  each  action  item,  mark 
an  “X”  in  the  appropriate  box  to  indicate  how  well  FSC  let  you  perform  (i.e.,  practice)  the  Item.  Then, 
circle  Yes  or  No  in  the  last  column  to  indicate  if  this  capability  should  be  kept  in  future  versions  of  FSC. 


ICCC  Action  Item 

Not  at  all 

Not  Very 
Well 

Moderately 

Well 

Very  Well 

Keep  in 

Future  Versions 

Analyze  civil  considerations. 

Yes  No 

Analyze  the  enemy  situation. 

Yes  No 

Apply  selected  defensive 
considerations  to  develop  a 
tactical  plan. 

Yes  No 

Apply  selected  steps  of  the 
troop  leading  procedure. 

Yes  No 

Apply  the  fundamentals  of 
conducting  a  movement  to 
contact. 

Yes  No 

Conduct  an  after  action  review 
for  a  light  infantry  company. 

Yes  No 

Conduct  mission  analysis. 

Yes  No 

Conduct  reconnaissance. 

Yes  No 

Develop  a  course  of  action  for 
a  light  infantry  company. 

Yes  No 

Determine  own  force  potential 
combat  power. 

Yes  No 

Integrate  fire  support  into 
urban  operations. 

Yes  No 

Integrate  selected 
fundamentals  and  specific 
planning  considerations  of 
the  offense  in  an  urban 
environment. 

Yes  No 

Integrate  the  fundamentals 
and  techniques  of  the  offense 
into  a  course  of  action. 

Yes  No 

Issue  a  company/team 
OPORD. 

Yes  No 

Issue  a  FRAGO. 

Yes  No 

Issue  OPORD  for  infantry 
company. 

Yes  No 

Perform  terrain  analysis. 

Yes  No 

Plan  breaching  operations. 

Yes  No 

Select  a  course  of  action. 

Yes  No 

Synchronize  a  light  company 
team  attack  in  an  urban 
operation. 

Yes  No 

Synchronize  the  engineer 
portion  of  a  light  infantry 
company  attack. 

Yes  No 

Synchronize  the  indirect  fires 
portion  of  a  light  infantry 
company  attack. 

Yes  No 

[Continue  on  next  page] 
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Parts.  -  Continued 


The  capability  to  perform  the  following  action  items  was  not  included  in  the  current  version  of  FSC. 
Indicate  by  circling  Yes  or  No  which  of  these  items  should  be  included  in  future  versions  of  FSC.  For 
those  Kerns  you  circle  “Yes,"  rank  order  the  five  action  items  that  you  believe  most  need  to  be  included  in 
future  versions  of  FSC.  The  most  important  Kern  should  be  assigned  a  *1"  and  the  least  important  Kern 
should  be  assigned  a  “5”. 


ICCC  Action  Item 

Putin 

Future  Versions 

Rank  Five 
Most 
Important 

Apply  emerging  threat  operations  to  tactical  planning. 

Yes  No 

Conduct  an  after  action  review  of  a  mechanized  company/team 
operation. 

Yes  No 

Conduct  TEWT. 

Yes  No 

Construct  a  company  METL. 

Yes  No 

Defend  a  company/team  battle  position. 

Yes  No 

Develop  a  communications  plan. 

Yes  No 

Issue  an  air  movement  and  landing  plan. 

Yes  No 

Issue  an  OPORD  for  a  mechanized  infantry  company. 

Yes  No 

Issue  engineer  portion  of  the  OPORD. 

Yes  No 

Issue  fire  support  portions  of  OPORD. 

Yes  No 

Perform  weather  analysis 

Yes  No 

Plan  a  hasty  air  assauK. 

Yes  No 

Plan  defensive  fire  support  for  mechanized  infantry 
company/team. 

Yes  No 

Plan  mobility  operations. 

Yes  No 

Plan  offensive  fire  support  for  a  mechanized  infantry 
company/team. 

Yes  No 

Plan  protective  obstacles. 

Yes  No 

Plan  survivability  operations. 

Yes  No 

[Continue  on  next  page] 
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Part  6.  Overall  Opinion  of  FSC 


1.  Does  high  quality  training  using  FSC  depend  on  an  experienced  instructor?  Circle:  Yes  or  No  . 
Can  you  learn  as  much  using  FSC  on  your  own?  Circie:  Yes  or  No 
Briefiy  indicate  the  basis  for  your  answers. 


2.  Identify  (a)  what  you  most  liked  and  (b)  what  you  most  disliked  about  the  following  components  or 
aspects  of  FSC.  For  your  most  disliked  feature  of  each  component  or  aspect  of  FSC,  what  improvements 
would  you  recommend? 

•  User  Manual 


•  Training  to  use  FSC 


•  Your  interface  with  FSC  during  the  mission  Planning  phase 


•  Your  interface  vrith  FSC  during  the  mission  execution  phase 


[Continue  on  next  page] 
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3.  What  did  you  like  gesi  about  FSC?  List  up  to  three  features  of  FSC  and  indicate  what  you  liked  about 
them. 


4.  What  did  you  like  JmsS  about  FSC?  List  up  to  three  features  of  FSC  and  indicate  what  you  disliked 
about  them. 


5.  In  spite  of  any  possible  limitations  you  may  have  encountered  wrfth  FSC,  what  are  the  advantages  or 
potential  of  FSC  for  improving  the  decision-making  skills  of  individuals  in  ICCC? 


Write  vour  additional  comments  and  suggestions  here  and  on  the  backside  of  the  cages 


[Thank  you  for  completing  this  questionnaire] 
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Janus  Scenario  Assessment  Checklist 
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12.  Provided  chain  of  command  with  accurate  ARTEP  7-10  MTP  (DRAFT).  Task  7-2-1000.  CO  Attack:  (1 )  Company  commander  assesses  and  reports  the  situation  to 

situation  reports  (SITREPS  conveyed  actual  higher  headquarters. 

situation). 
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Summary  of  Doctrinal  Errors  Committed  During  the  Janus  Exercise 

At  the  conclusion  of  each  mission,  notes  from  the  Janus  Scenario  Assessment 
Checklist  and  notes  taken  during  the  battle,  were  used  to  prepare  a  synopsis  of  the 
battle.  The  synopses  focused  not  only  on  the  flow  of  the  battle,  but  also  on  the 
doctrinal  errors  committed  by  each  Soldier. 

Once  the  synopses  were  completed,  the  observer/controller  reviewed  them  to 
identify  the  doctrinal  errors  that  occurred  during  each  mission.  A  list  was  prepared 
that  identified  common  errors  made  by  each  Soldier.  The  list  of  doctrinal  errors  for 
each  Soldier  was  annotated  and  added  to  each  synopsis.  The  table  below  identifies 
the  common  doctrinal  errors  that  were  considered. 


_ Common  Doctrinal  Errors _ 

Terrain  Analysis  Errors 

Did  Not  View  the  Battlefield  from  the  OPFOR  Commander’s  Perspective 

Overestimated  Movement  Speed _ 

Overall  Poor  Concept  of  the  Operation _ 

Failed  to  Isolate  the  Objective  (No  blocking  position  established) _ 

Failed  to  Adjust  Plan  to  Revised  Enemy  Threat _ 

Failed  to  Maintain  Mutual  Support  Among  Maneuver  Elements _ 

Failed  to  Provide  Timely  Supporting  Fires _ 

Failed  to  Sweep  Objective _ 

Failed  to  Adequately  Synchronize  Battlefield  Assets _ 
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Table  B-1 

Definition  and  Descriptive  Statistics  for  Indices  of  Military  Experience  Derived  From  Responses  to  Items  in 
the  Biographical  Information  Questionnaire 
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Table  B-2 

Descriptive  Statistics  for  Part  1  of  the  FSC  Questionnaire  -  Involvement  Items  (n=26) 
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Table  B-3 

Descriptive  Statistics  for  Part  2  of  the  FSC  Questionnaire  -  Overall  Training  Value  Items  (n=26) 


Table  B-4 

Descriptive  Statistics  for  Part  3  of  the  FSC  Questionnaire  -  Training  Value  of  Specific  FSC  Scenarios  Items 
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Table  B-5 

Descriptive  Statistics  for  Part  4  of  the  FSC  Questionnaire  -  FSC  Fidelity  Items 


Table  B-6 

Descriptive  Statistics  for  Part  5a  of  the  FSC  Questionnaire  -  Capability  to  Perform  ICCC  Action  Items  Using  FSC 
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Rank  of  Five  Most  Important  New  Capabilities 
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Item 

Indicate  if  the  capability  to  perform  the  following  ICCC 
action 

items  should  be  included  in  future  versions  of  FSC.  Then,  rank 
order 

the  five  that  most  need  to  be  included  in  future  versions  of 

FSC. 

Apply  emerging  threat  operations  to  tactical  planning. 

Conduct  an  after  action  review  of  a  mechanized  company/team  operation. 
Conduct  TEWT. 

Construct  a  company  METL. 

Defend  a  company/team  battle  position. 

Develop  a  communications  plan. 

Issue  an  air  movement  and  landing  plan. 

Issue  an  OPORD  for  a  mechanized  infantry  company. 

Issue  engineer  portion  of  the  OPORD. 

Issue  fire  support  portions  of  the  OPORD. 

Perform  weather  analysis. 

Plan  a  hasty  air  assault. 

Plan  defensive  fire  support  for  mechanized  infantry  company/team. 

Plan  mobility  operations. 

Plan  offensive  fire  support  for  a  mechanized  infantry  company/team. 

Plan  protective  obstacles. 

Plan  survivability  operations. 
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